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1 . INTRODUCTION AND OVERVIEW 



This monograph reports the results of a national siin-ey that examines 
institutional support for undergraduate student assessment. It provides a 
national profile of student assessment initiatives at doctoral and research 
institutions: advice for administrators, faculty, and staff who are designing 
student assessment approaches and support processes that are likely to fo.ster 
institutional improvement: and an instrument that in.stitutions can use to 
examine patterns for student assessment on their own campuses. 

This report is a result of the National Center for Postsecondary Improvement (NCPl ) 
research program examining organizational and administrative support for student assessment in 
postsecondary institutions. In 1997-98, NCPI researchers conducted a major literature review 
followed by a national survey on institutional support for student assessment. While other 
dissemination efforts have focused on a research audience, this report is specifically written for 
administrators, faculty and staff in doctoral and research universities who are involved with student 
assessment on their campuses. 



Purpose of the Monograph 

Administrators and faculty leaders have little credible and verifiable evidence to guide their 
planning and decision making regarding student assessment. The literature on student assessment 
offers many descriptions of student assessment practices at a variety of postsecondary institutions 
(Banta & Associates, 1993; Banta, Lund, Black, & Ohlander, 1996) and a number of limited 
surveys have collected information concerning student assessment measures and methods used by 
focused groups of institutions (Cowart, 1990; El-Khawas. 1988, 1989, 1990. 1991. 1992. 1993. 
1995, 1996; Johnson, Prus, Andersen, & El Khawas, 1991 ). There has been comparatively less 
consideration of how colleges and universities can develop an effective institutional strategy for 
as.sessment or of organizational and administrative practices that support internal engagement in 
assessment and enhance the use of student assessment data. Guidelines for planning and 
implementing student assessment efforts are available (American Association for Higher Education 
[AAHE], 1992; Ewell, 1988a, 1988b; Rossman & El-Khawas, 1987) hut largely missing is any 
systematic examination of the relationships among various ways institutions have approached and 
supported student assessment and the likelihood of their reaping positive impacts from student 
a.ssessment efforts (Banta et al., 1996; Ewell, 1988b; Gray & Banta, 1997). Furthermore, there 
has been little research focused specifically on student assessment within doctoral and research 
institutions. 

Understanding the national profile of undergraduate assessment activities and support 
patterns found in doctoral and research institutions is important for several reasons. First, it is 
helpful for institutions to know what their peers are doing in relation to student assessment. How 
are other institutions responding to external demands for student assessment? What approaches to 
student assessment are they using? How are they supporting and promoting student assessment 
institution-wide through their policies and practices? How are they using student assessment data? 
What impacts has student assessment had on faculty as well as institutional and student 
performance? Finally, if student assessment activities do make a difference in terms of improved 
teaching and learning on campuses, how does that happen? What external influences, institutional 
approaches, and organizational and administrative support practices encourage internal involvement 
in student assessment and positive impacts from student assessment data? 

This monograph will answer these questions through accomplishing three primary 
objectives. First, it will provide a national profile of current .student assessment practices and 
institutional support patterns within doctoral and research universities and compare this to 



assessment practices and support patterns found in all types of postsecondary' institutions. A 
second major objective is to provide advice for administrators, faculty, and staff in doctoral and 
research institutions on designing student assessment approaches and support processes that are 
likely to foster institutional improvement. Finally this report includes the sur\ey instrument used to 
gather data for this study in Appendix A. Institutions can use this inventory' to examine patterns for 
student assessment on their own campuses. 

We will discuss the student assessment methods used by institutions in this report, but it is 
not our intent to examine the specific instruments used or the measurement issues associated with 
their use. For a comprehensive review of the literature on student assessment in postsecondary 
institutions, readers are referred to Improving Organizational and Administrative Support for 
Student As.sessment: A Review of the Research Literature (Peterson. Einarson. Trice. & Nichols. 
1997). The data from this survey have been summarized in several forms including numerous 
conference papers and journal articles and a technical report entitled Institutional Support for 
Student Assessment: Methodology and Results of a National Survey (Peterson. Einarson. 
Augustine, & Vaughan, 1999). Readers who are interested in the statistical analyses conducted on 
the student assessment data should consult the technical report. While this report relies on those 
statistical analyses, it will not provide extensive statistical details. 

Definition of Student Assessment 

Our focus in this report is on undergraduate student assessment from an institutional 
perspective. We are interested in what approaches institutions u.se to assess student performance, 
how institutions are organized to promote and support student assessment, and how they use 
.student assessment to improve student, faculty and academic performance of the institution. 

In this monograph, we use the term student assessment to refer to activities other than 
traditional end-of-course grading that are used to measure a diverse array of dimensions of student 
performance or development. Institutions may decide to engage in student assessment for a variety 
of reasons or differing purposes. Cognitive, affective and behavioral dimensions of student 
performance and development may be assessed. Asse.ssment efforts may be directed toward 
.students as they enter the institution, during their enrollment or after they exit from the institution. 
Student assessment may use a variety of methods and may be planned and executed centrally 
within institutions or in a decentralized manner through the actions of individual academic units. 
This report will address all these dimensions of student assessment. 

Preview of the Monograph 

This report is presented in nine sections. Following this introduction, the second section, 
perspectives on student assessment, provides a brief recent histoi'y of the student assessment 
movement in general and in doctoral and research institutions specifically. The conceptual 
framework of institutional support for student assessment that was developed in the literature 
review (Peterson et al., 1997) and guided the development of the survey instrument is then 
presented. Section two ends with a brief summary of the methods used to conduct the research on 
institutional support for student assessment. 

Sections three through seven present the results of a national survey of institutional support 
for student assessment. These .sections parallel the domains in the framework. In each section, 
information is presented for both doctoral and research institutions and for all institutions that 
responded to the study. Section three focuses on institutions’ approaches to student assessment. 
This section is followed by a summary of the institution-wide support patterns evident in the study. 
Section five focuses on the influence of external groups, such as state agencies and accreditors, on 
institutional student assessment patterns. In this section, the influence of states on institutional 
student assessment activity is considered for public institutions only. Section six discus.ses 



assessment management practices and policies used by institutions to promote or assure the use of 
student assessment information. Section seven reports on the institutional uses and impacts of 
student assessment. 

Section eight is based on the statistical analy.ses conducted on the national sur\'ey data. It 
presents the findings of three sets of key relationships examined in the study. The first focuses on 
the influence of external groups on institutional approaches to student assessment. The second 
examines the relationship of external, influences, assessment approaches, institution-wide support 
patterns, and assessment management practices and policies to the institutional uses ot student 
assessment. The third analysis examines the relationship of external influences, assessment 
approaches, institution-wide support patterns, and assessment management practices and policies 
to the institutional impacts of engaging in student assessment. Section nine presents a summai'y ot 
our research findings and related recommendations for specific assessment activities within 
doctoral and research institutions. It concludes with general guidelines for institutions to use to 
examine and redesign or plan their student as.sessment process and functions. 



2. PERSPECTIVES ON STUDENT ASSESSMENT IN HIGHER EDUCATION 



An overview of the development of student assessment in hii>her education 
and in doctoral and research institutions suggests the need for the institutional 
perspective that guides this report. A guiding framework conceptualizes seven 
domains of an institution's student asses.sment strategy: institutional conte.xt. 
external influences, approach adopted, institutional support patterns, 
assesstnent management practices and policies, assessment culture and 
climate, and institutional uses and impacts of student asses.sment. 

This monograph concurs with the perspective of other scholars of student assessment. We 
agree that at its best, student assessment is not simply an exercise to gather data or respond to 
accountability requirements. Rather, an effective student assessment approach is one that gathers 
information about selected aspects of students' characteristics, achievements and experiences and 
u.ses this information to shape institutional policies, processes, and practices in ways that lead to 
improved student performance and institutional functioning (AAHE. 1992; Banta & Associates. 
1993; Ewell, 1984, 1987c, 1988b; Jacobi. Astin & Ayala, 1987). 

Growth of Student Assessment 

The assessment of student performance in higher and postsecondary education is not a new 
concept or phenomenon. The first College Board examination designed to assess student learning 
outcomes on a national scale was administered in 1901. The ensuing decades are marked by events 
that reflect a growing concern with assessing college student performance, such as; the emergence 
of a regional accreditation focus on student assessment; the establishment ot university-based and 
national testing centers; and the development of broadened taxonomies of student outcomes 
(Resnick & Goulden, 1987; Sims, 1992). However, these earlier developments pale in comparison 
to the emergence in the mid 1980s of student assessment as an important focus of educational 
policy at the national, state and institutional level — a focus that continues today. 

This heightened interest in student assessment is the result of many broad forces. In the 
larger .societal and political arena, concerns about consumer protection, the rising costs of 
education, the training and human resource needs for state and regional economic development, 
and fiscal pressures on state and federal government all have shaped public interest in the 
educational contribution of higher education institutions. At the institutional level, the challenges of 
expansion in the 1950s and 60s, the enrollment and financial constraints in the 1970s, and the new 
educational demands in the 1980s have shifted the managerial focus of perfomiance by higher 
education institutions from resource adequacy, to efficiency, to effectiveness, to broader concerns 
for academic and institutional quality. Consequently, over the past decade and a half the 
assessment and improvement of student perfomiance has been the focus of much of this discussion 
and of many efforts both external to and within colleges and universities. 

A variety of specific activities at the national level — reports, guidelines, legislation, 
educational goals, and funding — have been credited with providing the initial stimulus for the 
student asses.sment movement in higher education in the last decade and a half A series of national 
reports that critically examined the quality of education were published in the mid 1980s. The first, 
A Nation at Risk; The Imperative for Educational Reform (National Commission on Excellence in 
Education, 1983), prompted calls for reform in elementary and secondary education. The 
following year Involvement in Learning: Realizing the Potential of American Higher Education 
(National Institute of Education, 1984) addressed the conditions required for improving the quality 
of undergraduate education. Institutions were given three recommendations: .set high expectations 
for student learning, actively involve students in learning, and develop an institutionalized proce.ss 
for assessing student learning. Subsequent reports such as To Reclaim a Legacy; A Report on the 
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Humanities in Higher Education (Bennett, 1984) and Integrity in the College Curriculum: A Report 
to the Academic Community (Association of American Colleges, 1985) continued this focus on 
issues of undergraduate education quality and assessment. In 1988 the U.S. Department of 
Education reyised its Criteria for Recognition of Accrediting Agencies, stipulating that agencies 
must require educational institutions and programs to ( 1 ) clearly specify their educational objectiyes 
and (2) conduct student assessment to detemiine whether they were achieving these standards. In 
addition, the enactment of legislation such as the "Student Right to Know and Campus Security 
Act" and "Ability to Benefit" legislation (Education Commission of the States. 1991). adoption of 
the National Education Goals (Lenth, 1993, 1996: Nettles, 1995), and provision of federal funds 
for the development of institutions' assessment projects (Cook, 1989) contributed to the early 
momentum of the assessment movement. 

State governments, reflecting these national developments, have also played a key role in 
stimulating postsecondary student assessment. The first state-level mandate for student assessment 
appeared when the Florida state legislature directed the higher education .system to develop the 
College-Level Academic Skills Test in 1982. Since then, the number of states involved in student 
assessment has increased steadily with all but four of titty states now reporting some type ot 
student assessment initiative (Cole, Nettles, & Sharp, 1997). States have varied greatly in their 
approaches to student assessment policy, but there is evidence ot a general shitt in state-level 
approaches over the past decade. From the mid to late 1980s state assessment initiatives generally 
emphasized institutional improvement as their primary purpose and largely pemiitted institutions to 
design their own assessment efforts. Since 1990, states have placed greater emphasis on student 
assessment as a means of responding to extei'nal demands for accountability and have increasingly 
elected to mandate the content and form of institutions' assessment approaches. 

Regional accreditation associations also have emerged as an important inlluence on student 
as.sessment. Regional accrediting associations vary in the length of time they have been involved in 
student assessment, their specific reporting requirements, and the range of assessment-related 
services provided to member institutions (Cole et al., 1997). Since the federal government revised 
its criteria for recognizing accrediting agencies in 1988, all six regional accreditation associations 
have required member institutions to undertake and document some form of student assessment 
activity. In turn, a growing number of institutions have reported that accreditation requirements are 
an important reason for deciding to engage in and increase their student assessment efforts (El- 
Khawas, 1995; Johnson et al, 1991). 

Since the mid 1980s the number of postsecondai 7 institutions engaged in some form of 
student assessment activity has steadily increased (El-Khawas, 1988, 1990, 1995). However, by 
1990 only a small proportion of institutions had embarked on comprehensive student assessment 
programs (El-Khawas, 1990; Hexter & Lippincott, 1990; Johnson et al., 1991 ). Fewer still had 
reported achieving observable impacts from their student assessment efforts (Astin, 1991 ; 
Hutchings & Marchese, 1990). So it seems that, while many colleges and universities are investing 
faculty and administrative time and effort in student assessment, few are benefiting from its 
potential to improve student and institutional performance. 

Characteristics of Doctoral and Research Institutions 

Institutional type is an important dimension to consider when examining the approach an 
institution takes toward student assessment. To date, there has been little consideration of 
differences in effective organizational and admini.strative practices for promoting student 
assessment within specific types of institutions. Yet effective strategies and practices for planning, 
promoting and implementing undergraduate student assessment vary according to the type of 
postsecondary institution being considered — for example, what works in promoting student 
assessment in doctoral and research universities differs from what works in liberal arts colleges. In 
our national study of institutional support for student assessment, we found that there were many 



statistically significant differences by institutional type on the approaches to. support for. policies 
and practices regarding, and uses and impact of student assessment reported by institutions and 
that they were typically stronger than differences by other institutional characteristics such as size 
or whether the institution is public or private. 

The challenges doctoral and research institutions face as they develop student assessment 
programs are quite different from those faced by other institutions. These differences stem, in pan. 
from the institutional mission, curricular focus, organizational complexity, governance structure, 
faculty roles and responsibilities, and student clientele typically as.sociated with doctoral and 
re.search universities. 

Doctoral and research universities embrace a different institutional mission focus than other 
types of postsecondary in.stitutions. They are much more re.search-focused than two-year and other 
types of four-year institutions. They also place comparatively greater emphasis on graduate-level 
education. This emphasis on re.search and graduate education may affect the extent to which 
administrators and faculty members view improving undergraduate education through student 
assessment as an institutional priority, and so may influence their commitment to and involvement 
in assessment efforts. Building internal support for undergraduate student assessment may be more 
difficult to accomplish in doctoral and research universities than in other types of institutions that 
have a clearer focus on undergraduate teaching, such as liberal arts and associate of arts colleges. 

Doctoral and research universities tend to be the most organizationally complex in terms of 
the number of departments, schools, research enters, and administrative units (Dill. 1997). They 
are also typically large institutions. The combination of organizational complexity and institutional 
size may increa.se the challenges involved in developing an institution-wide asse.ssment program 
and obtaining adequate campus representation on assessment committees. 

The distribution of decision-making power between administrators and faculty members 
within these institutions differs from that found in other institutional types. Faculty in doctoral and 
re.search universities have greater professional authority and autonomy and are more actively 
involved in academic decision making than their peers in associate of arts and other types of four- 
year colleges (Blau. 1973; Clark, 1987; Dill, 1997). These differences in governance styles 
.suggest that effective patterns of organizational and administrative support for student assessment 
in doctoral and research institutions will be configured differently from those found in other types 
of institutions. Institutions with administratively-driven governance styles, such as associate of arts 
institutions, may be able to effectively use organizational policies to direct faculty involvement in 
assessment efforts while those institutions in which faculty have considerably more professional 
autonomy and decision-making power, such as doctoral and research institutions, may need to 
emphasize faculty leadership in planning and implementing assessment activities and require more 
time to establish an assessment program. 

Faculty in doctoral and research institutions are expected to focus on both research and 
teaching responsibilities. Despite recent efforts to raise the importance accorded to teaching as an 
institutional activity within these institutions, faculty still perceive it is the research role that 
continues to have primacy in decisions regarding faculty tenure and promotion (Gray, Diamond & 
Adam, 1996; Gray, Froh, & Diamond, 1992). On the positive side, faculty members’ familiarity 
with conducting research may make them quite comfortable with designing and implementing 
assessment methods and, analyzing the resulting data. Conversely, it could be that the willingness 
of faculty in doctoral and research institutions to engage in .student assessment is diminished by 
their emphasis on research. 

The characteristics of the student body at doctoral and re.search universities are also likely to 
be different from those at other types of institutions. Given their top position in the hierarchy of 
institutional .selectivity, doctoral and re.search institutions tend to attract students who are better 



academically prepared than those who enroll in comprehensive, less selective liberal arts, and 
associate of arts institutions. Compared to other types of institutions, doctoral and panicularly 
research institutions have a more traditional student body, enrolling more residential than commuter 
students and more full-time than part-time students. The high caliber of the student body may 
influence faculty and administrators’ views regarding the need for student assessment as well as 
the type of student competencies that should be assessed. 

The net effect of these distinctive characteristics may be to constrain the extent to which 
doctoral and research universities become involved in undergraduate student assessment, shape the 
patterns of organizational and administrative support developed to promote assessment, and limit 
the extent to which assessment information is used for institutional improvement. Several studies 
have tested and provide some support for these assertions. 

Scott (1991) found that administrators and faculty at doctoral universities had significantly 
less involvement in, knowledge about, and belief in the importance of their institution's assessment 
program than did comparable respondents from associate of arts and comprehensive institutions. 
Multi-institution studies by Ory and Parker (1989) and Jemmott (1992/1993) found that doctoral- 
granting institutions responded to external mandates for assessment in a superficial and compliant 
manner. A large-scale national survey conducted by the American Council on Education (El- 
Khawas, 1993) found these institutions were less involved in student assessment than other types 
of institutions. However, other studies offer conflicting evidence of the extent to which these 
institutions have been involved in undergraduate student assessment (Gentemann & Rogers, 1987; 
Smith, Bradley & Draper, 1993). Although the existing literature includes some single- and multi- 
in.stitution case studies of student assessment, it provides limited infomiation concerning the 
current asse.ssment activities conducted throughout the countiy by doctoral and research 
universities. Nor did we find research that examined the relationship of student assessment 
approaches and institutional support practices to using and achieving impacts from student 
a.s.sessment in doctoral and research institutions. 

A Framework of Institutional Support for Student Assessment 

An institution’s organizational and administrative context can be a powerful source of 
influence on administrators', faculty members' and students' behaviors and perceptions and can 
shape their views of what the institution values and which activities are important to engage in. 
Colleges and universities purposefully create a variety of student assessment approaches and 
structures, proce.sses, policies and practices to support and implement student assessment. 
Institutions have varied greatly in their approaches to assessing student performance and the ways 
in which they have organized to support their assessment efforts, a finding that is hardly surprising 
given the great diversity and relative autonomy of colleges and universities in the American 
postsecondary system. Yet there is little systematic evidence available concerning the specific 
manner in which post.secondary institutions have organized and administered to support student 
a.ssessment and the differential results of these efforts. For these reasons, we have chosen to 
examine student assessment from a re.search-based and institutional perspective. 

An extensive literature review identified what is currently known about the organizational 
and administrative context for student assessment in postsecondary institutions. A detailed 
description of the literature review process and findings is available in other publications (Peterson 
et al., 1999; Peterson et al., 1997). From this review, we developed a framework of institutional 
support for student assessment. This framework, displayed in Figure 1, consists of seven 
interacting domains. It conceptualizes how institutions respond to external pressures for student 
assessment, how they approach student assessment, the organizational and administrative patterns 
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they use to support student assessment, the assessment practices and policies they have adopted to 
promote student assessment, their culture and climate for student assessment, and how student 
assessment information is used by and has impacts on institutions. A seventh domain, institutional 
context, reflects the fact that these patterns probably differ significantly by institutional type, 
control and size. The culture and climate domain is not examined in this report and is the focus of 
future research activity. The other domains are briefly described below'. A description of the 
variables used in this study, by domain, is provided in Appendix B. 

External Influences on Student Assessment . A variety of external constituencies have 
played an important role in initiating and shaping student assessment efforts within postsecondaiy' 
institutions. In particular, state-level initiatives and regional accreditation associations may exeil 
strong direct influences through their requirements for student assessment. The business 
community, private foundations, and professional higher education associations may serve as 
weaker sources of influence — whether as participants in institutions' as.sessment efforts, as 
consultants or sources of resource materials, or as providers of funds for assessment-related 
activities. National efforts appear to have largely played an indirect role in influencing institutions' 
assessment efforts. 

Institutional Approach to Student As.sessment . Institutional approach to student assessment 
refers to institutional decisions regarding the collection and analysis of student assessment 
information. Important dimen.sions along which student assessment approaches can be 
differentiated include: the type or content of student assessment measures; the extent or array of 
student assessment data collected; the timing of data collection; the instmments and methods used 
to collect student assessment data; and the analy.ses conducted and reported for collected data. 

Organizational and Administrative Support for Student Assessment . The literature on 
student as.sessment identifies five important dimensions of organizational or administrative 
behavior patterns that may support student as.sessment efforts and enhance the likelihood that 
assessment will contribute to improvements in in.stitutional performance: the institution-wide 
support strategy for student assessment; administrative and governance activities used to promote 
student assessment; leadership and faculty support for student assessment; planning and 
coordination of student assessment; and procedures used to evaluate and revise student assessment 
efforts. 



A.ssessment Management Practices and Policies . Assessment management practices and 
policies refer to .specific practices, policies, procedures, or activities intentionally devised by 
institutions to implement and support their student assessment efforts. A number of content activity 
areas of this domain have been identified as potential influences on student assessment including: 
academic resource allocation; student inforrnation systems; internal access to student information; 
distribution of assessment reports and studies; student involvement in assessment; professional 
development for student asses.sment; faculty evaluation and rewards; and academic planning and 
review processes. 

Institutional U.ses and Impacts of Student Assessment . Ideally, institutions will use the 
information collected through student assessment processes to contribute to improvement in 
institutional and student performance. Three broad domains of institutional uses and impacts of 
student assessment information are suggested in the literature. These are the use of assessment 
information for institutional decision making, its impact on various internal constituents and 
phenomenon, and its impact on the institution’s relationships with its external constituents. 

Institutions may apply assessment information to several areas of academic decision 
making such as academic planning and review, the academic mission and goals, internal re,source 
allocation, and faculty evaluation and rewards. Student as.se.ssment infomiation may have both 
internal impacts, such as stimulating faculty intere.st in teaching and enhancing student learning. 






and external impacts, such as influencing state funding or re-accreditation decisions. The primaiy 
concern of this framework is to examine the relationship of external influences, institutions* 
as.se.ssment approaches, patterns of organizational and administrative support for assessment, and 
assessment management practices and policies to institutional uses and impacts of assessment 
information. 

In.stitutional Context . Broad in,stitutional characteristics such as institutional type, control 
and size are expected to moderate external influences on assessment, the institutional approach to 
student as.sessment, organizational and administrative support patterns, assessment management 
practices and policies, and institutional uses and impacts of as.sessment information. 

Description of National Survey 

Based on our review of the .student as.sessment literature, we developed a survey 
instrument or inventory called "Institutional Support for Student Assessment" (ISSA). The 
in.strument focuses on the as.sessment of undergraduate students conducted by postsecondary 
institutions ^ — not by individual faculty or academic sub-units within institutions. The instrument is 
designed as a comprehensive inventory of external influences on institutions' student assessment 
efforts, institutional approaches to student as.sessment, organizational and administrative support 
patterns for student assessment, assessment management practices and policies used, and 
institutional u,ses and impacts of student assessment information. The instrument was intended to 
a.ssist institutions in obtaining a clear picture of their own student assessment efforts, to provide a 
national profile of undergraduate student assessment efforts in postsecondary institutions, and to 
increase current understanding of how institutions can engage in and promote student assessment 
that produces positive impacts on academic, .student and institutional perfomiance. A copy of the 
survey instrument is included in Appendix A. 

In January 1998, the ISSA instrument was mailed to the chief academic administrator at all 
po.stsecondary institutions recognized by the U.S. Office of Education that offer undergraduate 
programs at the associate or baccalaureate degree level, excluding proprietary and specialized 
institutions. Completed .surveys were received from 1,393 of the 2,524 institutions meeting these 
criteria for an overall response rate of 55%. Doctoral and research colleges and universities were 
well represented in survey responses. Of the 229 doctoral and re.search institutions to whom 
in.struments were mailed, 145 or 63% completed and returned the survey. More than half of 
eligible public and private doctoral and research institutions participated. Survey responses were 
quite evenly di.stributed across states and accrediting regions. Given this strong and diversified 
respon.se rate, we are able to offer a repre,sentative profile of undergraduate student assessment 
activities undertaken in doctoral and research colleges and universities. Throughout this report, we 
compare .student as.sessment activities within doctoral and research institutions to student 
asse.ssment activities within all institutions responding to our .survey, including doctoral and 
research institutions. 
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3. THE INFLUENCE OF EXTERNAL GROUPS 



Examines state and regional accreditation association requirements and 
external sources of support for student assessment. While both states and 
regional accrediting agencies have influenced institutions to engage in .student 
assessment, accrediting associations are reported to have greater influence 
than states on doctoral and research institutions’ assessment initiatives. 
Professional associations are the most frequently used e.xtenial .source of 
support for student assessment. 

State officials and regional accreditation associations have increasingly required 
postsecondary institutions to become engaged in student assessment. Together with professional 
associations and private foundations, they have also offered various services to support 
institutions’ student assessment efforts. There has been relatively little examination of institutions' 
perceptions of and experiences with these external groups. To examine this domain, our survey 
included questions concerning: state requirements for student assessment; regional accreditation 
association requirements for student assessment; and external sources of support for student 
assessment. The following sections address each of these areas in turn. 

State Role in Student Assessment 

States have varied widely in the development and content of their initiatives for 
postsecondary student assessment (Cole et al., 1997). The survey asked for institutions' 
perceptions of three dimensions of state assessment plans: the development process and reporting 
requirements of state assessment plans; the influence of state requirements on institutions' 
assessment efforts; and state review of institutions' assessment plans or processes. The questions 
in this section were directed to state-funded institutions only. 

Development and Reporting Requirements of State Assessment Plans 

Development . Scholars have suggested that the influence of state assessment initiatives on 
institutions’ as.sessment efforts may depend on whether or not institutions have had input in the 
development of the initiatives. Survey respondents were asked to indicate whether their state's plan 
for student as.sessment was developed primarily by state officials, through joint consultation 
between state officials and institutional representatives or whether no state plan or requirement for 
student assessment existed. Responses from public doctoral and research universities and all 
institutions receiving state funding are displayed in Figure 2. 

Figure 2. Development of State Plan for Student Asse.ssment 
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Half of the public doctoral and research institutions in our study report that a state plan or 
requirement for student assessment exists. Institutions more often report having a state plan tor 
assessment that was developed in consultation between state and institutional officials (369r) than 
by state officials alone (13%). Public doctoral and research institutions differ little from all public 
institutions in their perceptions of the existence and development of state plans for student 
assessment. 

Reporting Requirements . Scholars also contend that the influence of state as.sessment 
initiatives will vary with the specific nature of the reporting requirements as.sociated with the.se 
initiatives. What types of information are institutions required to report to state officials? We asked 
institutions with state plans for assessment which of the following types of information they were 
required to report as part of their state’s assessment plan: evidence ot a student as.sessment plan; 
measurement of state-mandated student performance indicators; measurement of institutionally- 
developed student performance indicators; and evidence of having used student assessment 
information. Institutions checked all applicable reporting requirements. Respon.ses from doctoral 
and research universities and all institutions receiving state funding are displayed in Figure 3. 

Figure 3. State Reporting Requirements for Student Assessment 
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The mo.st common reporting requirement.s for public doctoral and re.search in.stitution.s are 
evidence of a student assessment plan (61%), state-mandated student performance indicators 
(60%), and institutionally-devised .student performance indicators (61%). A much smaller 
proportion (39%) are required to provide evidence of having u.sed .student a.ssessment information. 
Compared to all public institutions in our study, public doctoral and research institutions are more 
likely to report on institutionally-devised measures of student performance and 
less likely to provide evidence of assessment information use. This emphasis on institutionally- 
developed student performance indicators may promote institutional support for .student 
assessment. However, the low emphasis on providing evidence of assessment information use 
may reduce the likelihood that state requirements will encourage doctoral and research institutions 
to apply assessment results to institutional improvement. 






Influence of State Assessment Requirements 

What influence have state assessment requirements had on institutions' undergraduate 
student assessment activities? Institutions were asked which of the following impacts on their 
assessment efforts they attributed to state assessment requirements: were an important reason for 
institution initiating undergraduate student assessment: increased institution's involvement in 
undergraduate student assessment; were not a factor in institution's undergraduate student 
assessment activities; or were a negative influence on institution's undergraduate student 
assessment activities. Institutions selected all applicable influences. Responses from public 
doctoral and research universities and all public institutions are displayed in Figure 4. 



Figure 4. Influence of State Requirements for Student Assessment on Institutions' 
Assessment 
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*Only institutions with a state plan for student assessment responded to this question. 



Public doctoral and research institutions most often report positive influences of state 
requirements on their student assessment activities. Almost two-thirds (61%) report state 
requirements have increased their institution’s involvement in undergraduate student assessment 
and two-fifths (43%) indicate state requirements played an important role in their institution 
initiating student assessment efforts. Less than one-quai'ter (22%) believe state requirements have 
had no influence on their institution’s student assessment efforts while only 6% indicate state 
requirements have been a negative influence. Survey respondents from public doctoral and 
research universities do not have appreciably different perceptions of the influence of state 
assessment requirements than do respondents from all public institutions in our study. 

State Review of Institutions’ Student Assessment Plans 

Occurrence of State Review . States vary in terms of whether and how they evaluate 
institutions’ student assessment plans or information (National Center for Higher Education 
Management Systems, 1996). The survey asked respondents whether state officials had reviewed 
or evaluated their institution’s student assessment plans or processes. If a state review or 
evaluation had occurred, respondents indicated whether this review had been conducted by state 
officials, external reviewers or the institution itself. Responses from public doctoral and research 
universities and all public institutions are presented in Figure 5. 
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Figure 5. Methods Used to Review Institutions’ Student Assessment Plans or Processes* 
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*Only institutions with a state plan for student assessment responded to this question. 

Slightly less than half of public doctoral and !e.search institutions report their undergraduate 
student assessment plan or process has been reviewed by their state higher education agency 
(46%). Public doctoral and research institutions are much less likely than all public institutions to 
have undergone a state review of their undergraduate student as.ses.sment plan or process. More 
than half (56%) report that their state higher education agency has not conducted such a review. 
When a state review has occurred, it has most often been conducted by state officials (38%) and 
less often by external reviewers (14%) or the institution it.self (17%). 

State Review Criteria . Institutions that had undergone state review of their student 
assessment plan or process specified if the evaluation: reviewed the institution's student 
asse.ssment process; compared the institution's student performance record to its past performance; 
compared the institution’s student performance record with that of peer institutions; or compared 
the institution’s student performance record with thatiof other institutions in the state. Respon.ses 
from public doctoral and research universities and all public institutions are pre.sented in Figure 6. 

When public doctoral and research institutions have undergone a state review of their 
student assessment plan or process (44% of our doctoral and re.search respondents), this review 
has most often focused on the institution's student as.ses.sment process it.self (66%). Comparisons 
of institutions’ student performance records are less common. If conducted, the.se most often 
involve comparisons of the institution’s student perfomiance to other institutions in the same state 
(54%), followed by comparisons to their own past performance (5 1 %) or, least often, to the 
student performance records of peer institutions (34%). Compared to all public institutions in our 
study, public doctoral and research universities are much more likely to have their current student 
performance record compared with other institutions in the state. No other differences in state 
review criteria are evident. 

Regional Accrediting Role in Student Assessment 

Regional accreditation agencies have been reported as important influences on institutions' 
decisions to begin or expand their student assessment activities (El-Khawas, 1990, 1992, 1995; 
Johnson et al., 1991). The majority of doctoral and research institutions participating in our study 
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Figure 6. State Review Criteria for Institutions’ Student Assessment Plans or Processes* 
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*Only institutions with state plans for student assessment responded to this question. 



(78%) have completed a regional accreditation review which required undergraduate student 
assessment. To examine the role of regional accreditation associations in student as.sessment. the 
.survey a.sked about the reporting requirements for regional accreditation and institutions' 
perceptions of the influence of regional accreditation requirements on their a.s.ses.sment efforts. 

Regional Accreditation Reporting Requirements for Student Assessment 

Regional accreditation associations vary in terms of the evidence they require institutions to 
report about student assessment efforts. Accreditation requirements to report on the u.se being made 
of assessment information may contribute to greater institutional impacts from assessment than 
requirements that ask merely for evidence of an assessment plan or as.se.s.sment data. We a.sked 
institutions which of the following types of information they were required to report to their 
regional accreditation as.sociation: evidence of a student assessment plan; intended institutional u.ses 
of student assessment information; results of student as.sessment; and evidence of having used 
student assessment information. Institutions checked all applicable reporting requirements. Few 
(9%) doctoral and research institutions were unfamiliar with their regional accreditation 
requirements for student assessment. Re.sponses from doctoral and research universities and all 
institutions are displayed in Figure 7. 

The majority (83%) of doctoral and re.search institutions must provide their regional 
accreditation as.sociation with evidence that a student a.s.ses.sment plan or process is in place. A 
large proportion are required to provide evidence of the u.se they have made (70%) or intend to 
make (68%) of student as.se.ssment information. Slightly fewer report having to provide the results 
of student assessment (60%). The regional accreditation reporting requirements for student 
assessment reported by doctoral and re.search institutions do not differ much from those reported 
by all institutions in our study. 

Influence of Regional Accreditation Association Requirements for Student A.s.sessment 

What influence have regional accreditation as.sociation requirements had on institutions’ 
undergraduate student assessment activities? Institutions were a.sked which of the following 
impacts on their assessment efforts they attributed to their regional accreditor’s as.sessment 
requirements: were an important reason for initiating undergraduate student a.s.sessment; 




Figure 7. Regional Accreditation Association Reporting Requirements for Student Assessment 
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increased institution's involvement in undergraduate student assessment: were not a factor in 
institution’s undergraduate student assessment activities: or were a negative influence on 
institution’s undergraduate student assessment activities. Institutions selected all applicable 
influences; Responses from doctoral and research universities and all institutions are displayed in 
Figure 8. 



Figure 8. Influence of Regional Accreditation Requirements for Student Assessment on 
Institutions" Assessment Activities 
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Three-quarters (77%) of doctoral and research institutions report regional accreditation 
requirements have led to an increase in their institution's involvement in student assessment. Only 
half (50%) state these requirements were an important reason for their institution to initiate its 
undergraduate student assessment activities while almost one-fifth (18%) report that regional 
accreditation requirements have had no influence on their institution’s assessment efforts. Virtually 
none (1%) report negative consequences on their a.ssessment efforts stemming from these 
requirements. While doctoral and research institutions generally report positive influences from 
regional accreditation requirements for student assessment, they are somewhat less positive in their 
perceptions of accreditation influences than are all institutions in our study. In particular, they are 
much less likely to credit accreditation requirements with initiating institutional asses.sment efforts. 



External Sources of Support for Student Assessment 



External constituents may influence institutions' involvement in student assessment by- 
providing funding or serv'ices to support assessment efforts. Little is known about the extent to 
which institutions have availed themselves of these external sources of support. Our surv'ey asked 
institutions about their use of external grants and other services for improving their student 
assessment efforts. 

Receipt of External Grants for Student Assessment 

Institutions were asked if they had received grants to improve or support their student 
assessment practices from any of the following external sources; Fund for the Improvement of 
Postsecondary Education (FIPSE), other federal agencies, state incentive programs, and private 
foundations or corporate sources. Responses from doctoral and research universities and all 
institutions are presented in Figure 9. 

Figure 9. Receipt of External Grants for Student Assessment by Source 
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The majority of institutions, both doctoral/research and all, have not received external 
grants for improving student assessment practices (74% and 79% respectively). When doctoral and 
research institutions have received grants, these are most likely to have come from private 
foundations or corporate sources (11%) and FIPSE (10%). However, no more than 1 1% of 
doctoral and re.search universities report receiving as.sessment-related grants from any of the 
external sources considered in this question. It is unclear whether this low rate of grant receipt 
stems from unavailability, ineligibility, or lack of awareness of external funding opportunities. 

Use of External Services to Support Student Assessment 

A variety of postsecondary organizations — professional associations, regional accrediting 
associations, state-level agencies, and consortia of institutions — provide a range of services 
intended to support institutions’ student assessment efforts. These services include consultation, 
assessment conferences, training workshops, and publications or research reports on student 
assessment. For each of these four types of postsecondary organizations, the survey asked 
institutions which, if any, of these student assessment services they had u.sed. Responses from 
doctoral and research universities and all institutions in our study are presented in Table 1. 
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Doctoral and research institutions' use of assessment-related services varies by the type ot 
external service provider. They are more likely to use services from professional associations, and 
to a lesser extent, from regional accreditation associations than from state agencies or institutional 
consortia. They also use certain types of assessment support ser\'ices more than others, regardless 
of .service provider. Doctoral and research universities make most frequent use of conferences and 
publications or research reports on student assessment. A smaller proportion have used training 
workshops from one or more providers while consultation services are least likely to have been 
used. Compared to all institutions in our study, doctoral and research universities make much 
greater u.se of publications and conferences on asse.ssment provided by professional associations 
and le.ss use of as.sessment conferences provided by regional accreditation associations. 

Summary Observations 

The previous sections have addressed the influence of external constituencies, particulaiiy 
state agencies and regional accreditation associations, on doctoral and research institutions' 
undergraduate student assessment efforts. State-level efforts clearly play an important role in 
student as.sessment. Half of the public doctoral and research institutions in our study report they 
are subject to some form of state plan or requirements for undergraduate student assessment. 
As.sessment plans are more likely to have been developed through the joint efforts of state and 
institutional representatives than by state officials alone. Based on our respondents’ reports, there 
is a tendency for state reporting requirements to be accountability-oriented — more often requiring 
evidence of institutional assessment plans or student performance measures than institutional uses 
of as.sessment information. In contrast to earlier studies (Ory & Parker, 1989; Jemmott. 

1992/1993), public research and doctoral re,spondents are as likely to perceive their institution as 
having been positively influenced by external mandates for student assessment as are respondents 
from all types of public institutions. However, they are le,ss likely than other types of institutions to 
have undergone a state review of their student assessment plan or process. 

Compared to state-level efforts, regional accreditation associations appear to be a source of 
even stronger and more positive support for doctoral and research institutions' engagement in 
student assessment activities. According to survey respondents, regional accreditors more often 
require evidence of institutions’ intended or actual uses of student as.sessment information than do 
state reporting requirements. This pattern of reporting requirements may be more conducive to 
building institutional support for and use of student assessment than the more accountability- 
oriented approaches employed by state-level officials. Doctoral and research institutions are le.ss 
likely than all institutional types to perceive accreditation requirements as a reason for beginning 
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their assessment efforts. But overall, compared to state assessment requirements, regional 
accreditation requirements are more often viewed as having been a positive influence and less often 
reported as having had no influence or a negative influence on institutions' engagement in 
assessment. These findings support past research concerning the strong role of regional accrediting 
associations in promoting student assessment in postsecondary institutions. 

Finally, doctoral and research institutions have made varied use of potential external 
sources of support for student assessment. Most have not received external funds to improve their 
student assessment efforts. They are more likely to make use of assessment-related services 
provided by professional associations, particularly conferences and publications on student 
assessment. Services from regional accreditation associations. in.stitutional consortia and state 
agencies are used less often. Given the strong disciplinary affiliation typically maintained by 
faculty in doctoral and research universities, it is logical that professional associations offer the 
most appropriate external venue of support for student assessment. 



4. APPROACHES TO STUDENT ASSESSMENT 






Addresses die Type or content of student assessment measures used hy 
doctoral and research institutions, the timing of those assessments, their ii.'ic 
of standardized instruments and less traditional student assessment methods, 
and types of assessment studies conducted and student petfonnance reports 
produced. With the exception of basic college-readiness skills, doctoral and 
research institutions collect more extensive data on affective or behavioral 
measures than cognitive measures, and from current rather than former 
students. Tests and instruments are used more often than less-traditional 
assessment methods. Compared to all institutions, doctoral and research 
universities more often conduct various analyses of collected assessment data. 
Assessment reports are most often prepared for the institution as a whole and 
for schools or colleges within the institution. 

Assessment approach decisions may be influenced by external mandates, institutional 
context, and domains of the organizational and administrative environment. An institution's 
assessment approach may shape the uses and impacts of assessment information. Assessment 
scholars contend that institutions that collect comprehensive student assessment infomiation — 
those that collect data on various dimensions of student performance, at multiple points in time, 
through a variety of assessment methods — and that conduct and repon a variety of analyses of 
as.sessment data are more likely to use and achieve positive impacts from student assessment data. 

What approaches to student assessment have doctoral and research universities adopted? 
The ISSA instrument addressed the type or content of student assessment measures used by 
institutions, the timing of those assessments, their use of standardized instmments and less 
traditional student assessment methods, and types of assessment studies conducted and student 
performance reports produced. Here we examine the profile of student assessment approaches in 
doctoral and research institutions and compare it to the profde for all types ot postsecondaiy 
institutions. 

Type and Extent of Student Data Collected 

Data Collected from Current Students 

What types of data do institutions collect from their students? Survey respondents reported 
the extent to which their institution collects data on ten types of student data for cuirently enrolled 
students and four measures for former students. Figures 10, 1 1 and 12 display mean scores for the 
extent to which doctoral and research universities and all institutions collect these data. 

Cognitive Data . Figure 10 shows the extent to which institutions collect five types of 
cognitive data: basic college-readiness skills, higher-order skills (for example, critical thinking and 
problem solving), general education competencies, competence in major field of study (discipline- 
or program-specific knowledge), and vocational or professional skills. The latter four of these five 
types of data factored together to create a “cognitive assessment” index that we used in regression 
analyses presented in section eight. 

Doctoral and research institutions collect extensive data on students’ basic college-readiness 
skills. The mean score for this measure (3.16) indicates these data are collected for many to all 
students. Doctoral and research universities collect comparatively less data on other measures of 
their current students' cognitive performance. Data on students’ competencies in their major field 
and general education are collected from some to many students (mean scores of 2.53 and 2.42 



O 

ERIC 



20 



28 



Figure 10. Extent of Cognitive Data Collected on Current Students 
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respectively), but most doctoral and research universities collect data on vocational and higher- 
order skills from only some of their students (mean score of 1.96). They do not differ appreciably 
from all respondent institutions in their extent of data collection for these cognitive measures. 

Affective and Behavioral Data . Figure 1 1 presents five types of affective and behavioral 
data collected from currently enrolled students: personal growth and affective development, 
experiences and involvement with the institution, satisfaction with the institution, academic 
intentions, and academic progress. The first three of these items factored together into an “affective 
assessment” index that we used in regression analyses presented in section eight. 



Figure 11. Extent of Affective and Behavioral Data Collected on Current Students 
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Doctoral and research institutions collect data on the academic progress of vimially all their 
students (mean score of 3.91). This type of data is a common component of routine internal 
record-keeping. However, they are slightly less likely to collect data on students' academic 
intentions (mean score of 3.1 1). It is probably easier to monitor academic progress via transcripts 
and other institutional data than it is to collect and update information on students' academic goals. 
Doctoral and research institutions ask many students about their satisfaction with the institution 
(mean score of 2.84) but collect comparatively less information on aspects of their students' 
experiences and involvement within the institution (mean score of 2.69) and students' personal 
growth and affective development (mean score of 2.43). Compared to all in.stitutions participating 
in our study, doctoral and research universities place more emphasis on assessing students' 
personal growth and affective development. 

Data Collected from Former Students 

Figure 12 displays mean scores for the extent to which doctoral and re.search institutions 
and all institutions collect four types of data from former students: vocational or professional 
outcomes (for example, job attainment or performance), further education (for example, transfer, 
degree attainment, graduate .study), satisfaction and experiences with the institution after leaving, 
and civic or social roles (for example, political, social or community involvement). These latter 
three items factored together to create a “post-college" index used in regression analy.ses pre.sented 
in section eight. 

Figure 12. Extent of Data Collected on Former Students 
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Doctoral and research institutions report moderately extensive collection of data on three 
dimensions of former students’ performance or development. Measures of vocational outcome.s, 
further education and satisfaction with the in.stitution are collected from some to many of their 
former students (mean scores of 2.5 1, 2.48 and 2.54 respectively). However, they are less likely 
to collect data concerning former students’ civic or social roles, doing so for only some .students 
(mean score of 1.94). This lower emphasis on civic and .social outcomes may reflect the evaluation 
criteria applied to these institutions by a variety of constituencies. Prospective students and their 
parents, employers, accreditors and state-level officials are more likely to evaluate doctoral and 
research institutions’ performance on the basis of whether or not their undergraduates attain field- 





related employment, are accepted for graduate or post-graduate studies, and are satisfied with their 
educational experiences than on undergraduates' citizenship skills and activities. 

The profile of data collection from former students by doctoral and research institutions 
closely resembles the profile reported by respondents from all types of institutions. With the 
exception of civic or social roles data, doctoral and research institutions collect less extensive data 
from their former students but these differences are not of practical significance. 

Timing of Data Collection 

Researchers have stressed that it is particularly important for institutions to collect student 
assessment data on students at varying points in time in order to assure that students are meeting 
their academic goals and that the college experience is “value-added." When are doctoral and 
research institutions collecting student assessment data? Table 2 presents the timing ot data 
collection efforts for each of nine measures of student performance for doctoral and research 
universities and all institutions. Institutional respondents told us whether they collect data for each 
measure at entry, during students’ enrollment, and/or at exit. Respondents could choose all three ol 
these options for each type of data. 



Table 2. Institutions 
Data at Entry, While 


Collecting Cognitive, Affective, and Behavioral Student 
Student is Enrolled, and at Exit 


Assessment 




9r 


Institutions Collecting Data 






At Entry 


While Enrolled 


Al Exit 


Data Collected 


Doc/Res All 


Doc/Res All 


Doc/Res All 


Cognitive Data 


Basic Skills 


96 97 


20 21 


6 8 


Higher Order Skills 


27 32 


79 66 


27 33 


General Education 


27 32 


72 64 


26 36 


Major Competence 


7 9 


69 62 


60 64 


Vocational Skills 


5 10 


72 67 


60 60 


Affective and Behavioral 
Data 


Personal Growth 


45 32 


74 67 


42 41 


Student Involvement 


14 7 


73 72 


59 55 


Student Satisfaction 


14 7 


73 71 


67 65 


Academic Intentions 


88 92 


44 36 


28 19 



• Timing of Collecting Cognitive Data 

With the exception of basic skills data, most cognitive data are collected either while the 
student is enrolled or at exit. It is logical that most doctoral and re.search institutions that collect 
basic skills data do so at the point of entry into the institution (96%). Roughly three-quarters of 
doctoral and research institutions that collect data on students’ higher-order cognitive skills and 
^ general education competencies do so while the student is enrolled (79% and 72% of institutions 

respectively). This is greater than the proportion of all institutions doing so in our study. However, 
only one-quarter of doctoral and research institutions assess these cognitive data at point of exit 
(27% and 26% respectively), a smaller proportion than reported among all institutions. 

More than two-thirds of doctoral and research institutions collect data on competencies in 
^ the major and professional/vocational skills during students’ enrollment (69% and 72% 

respectively). A large proportion report collecting these types of data from their exiting students 
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(60% each). At first glance, these findings suggest that doctoral and research institutions are 
assessing their students’ major competencies and vocational skills at more than one point in time. 
But other analyses (not shown in Table 2) show that less than one-quarter (24%) ot doctoral and 
research institutions are collecting major competence data from their students both while they are 
enrolled rmr/ at exit, and less than one-fifth (18%) do so for vocational/professional skills. The 
remaining institutions are collecting major competence and vocational skills data ci7//cr while their 
students are still enrolled or at exit. 

Timing of Collecting Affective and Behavioral Data 

With the exception of academic intentions, most doctoral and research institutions collect 
affective data either during enrollment or at exit. In terms of academic intentions, it again is not 
surprising that students’ intentions are collected at entry (88%). But these intentions are less often 
a.sse.s.sed at subsequent points during students’ enrollment even though students' goals may 
conceivably change throughout their time at an institution. While a large proportion of institutions 
report collecting data on students’ personal growth, involvement and satisfaction with the 
institution during enrollment and at exit (from 42% to 74%). only \6% of institutions report 
collecting personal growth data both while students are enrolled and at exit and only one-quarter 
report doing so for student involvement and satisfaction (23% and 28% respectively). A larger 
proportion of doctoral and research institutions than all institutions report collecting data on 
entering students’ personal growth and affective development. There are no other notable 
differences between the timmg of doctoral and research institutions' data collection efforts and 
tho.se of all institutions in our study. 

Student Assessment Instruments and Methods Used 

Institutions engaging in student assessment efforts must select the means by which 
asse.ssment data will be collected. A traditional choice is to use assessment instruments, generally 
in the fomi of objective examinations or inventories administered in a pencil and paper or 
computerized format. More recently, there has been growing interest in alternative methods ot 
a.s.se.ssing students. These non-traditional methods tend to be more qualitative or integrative in 
approach — requiring students to demonstrate higher-order cognitive skills such as application or 
.synthesis — or may use sources of information other than students themselves. Further, as the, 
student body entering postsecondai'y institutions becomes more diverse through the increased 
enrollment of part-time, older and minority students), some scholars have suggested the need to 
use special a.s.sessment methods to reflect the unique characteristics, learning styles, needs and life 
situations of various sub-populations of students. 

Student Asse.ssment Instruments 

How are institutions collecting student assessment data? Survey respondents reported 
whether their institution uses instruments or tests to collect any of ten types of assessment 
information: nine for current students (basic college-readiness skills; higher-order skills; general 
education competencies; competence in major field of study; vocational or professional skills; 
personal growth and affective development; experiences and involvement with the institution; 
satisfaction with the institution; and academic intentions) and one for former students (alumni 
satisfaction and experiences). If an instrument was used, respondents specified its source — 
institutionally-developed, state-provided, or commercially available. Figures 13 and 14 present 
information on these eleven types of assessment information collected via traditional instruments or 
tests for doctoral and research institutions and all institutions in our study. 



Figure 13. Institutional Use of Student Assessment Instruments to Collect Cognitive Data 
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Figure 14. Institutional Use of Student Assessment Instruments to Collect Affective and 
Behavioral Data 
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As Figures 13 and 14 show, the use of assessment instruments or tests varies with the type 
of assessment data being collected. Doctoral and research institutions are most likely to use 
instruments or inventories to collect data on alumni and .student satisfaction (97% and 95% 
respectively), student involvement (86%), and basic college-readiness skills (82%) and least likely 
to use them to collect data on students’ higher-order cognitive skills (56%), vocational or 
professional skills (59%), and general education competencies (60%). 

When a test or instrument is used, it has most often been developed by the doctoral and 
research institution itself (data concerning the source of instruments for each type of data are 
available in Appendix A of this report). This is especially true for instruments measuring aspects of 
student performance or experiences that are specific to the institution such as alumni satisfaction, 
student satisfaction, student involvement with the institution, and competence in the major. In 
addition, doctoral and research institutions frequently use commercially-available instruments for 
measuring academic intentions and higher-order cognitive .skills. These measures are both more 




uniform across campus settings and. particularly for higher-order skills, more complex to develop. 
Doctoral and research universities make limited use of state-provided instruments (no more than 
13% of institutions reported using a state-provided in.strument for any of the types of data 
considered), suggesting that such'instmments are not a part of state-level requirements for student 
assessment. 

Doctoral and research universities do differ from all institutions in a few aspects of their use 
of as.sessment instmments and tests. Compared to all institutions, they make greater use of 
institutionally-developed instmments to collect data on students' basic college-readiness skills and 
personal growth and affective development; they more often use commercially-available 
instmments to collect data on students’ academic intentions; and less often use commercially- 
available instmments to measure students’ basic college-readiness skills. 

Other Student Assessment Methods 

Institutions afso reported the extent to which they use each of nine alternative methods of 
student assessment. Table 3 presents this infomiation for doctoral and research universities and all 
institutions in our study. 



Table 3. Mean Scores of Extent of Use of 


Other Student Assessment 


Methods 


Sludenl Assessment Methods 


Doctoral and Research 
(n=l4.'^) 


All Institutions (n=1.393) 


Student-Centered 


Observations of student peribrmance 


2.12 


2.26 


Student portfolios or comprehensive projects 


2.13 


2.10 


Student performance in capstone courses 


2.27 


2.15 


Student interviews or focus groups 
Extemallv-Oriented 


1.99 


1.84 


Alumni interviews or focus groups 


1.96 


1 .90 


Employer interviews or focus groups 


1.84 


1.87 


Other 


Transcript analysis 


1.92 


2.16 


External examination of students 


2.02 


2.02 


Surveys or interviews with withdrawing students 


2.18 


2.40 



I = not used; 2 = used in sonic units; 3 = used in most units; 4 = used in all units 



Four of these options require more active participation of currently enrolled students and 
factored together to create a “student-centered” index of assessment methods that were used in 
regression analyses presented in section eight: observations of student performance, student 
portfolios or comprehensive projects, capstone courses, and student interviews or focus groups. 
None is u.sed extensively by doctoral and research institutions. Of these four methods, doctoral and 
research institutions are most likely to use capstone courses (mean score of 2.27), portfolios or 
comprehensive projects (mean score of 2.13) and observations of .student performance (mean score 
of 2.12) — doing so in some to most units or departments. 

Two of these alternative measures rely on information from external groups or sources and 
factored together to create an index of “external methods” of student as.sessment that we used in 
regression analyses in section eight: alumni interviews or focus groups, and employer interviews 
or focus groups. Doctoral and research institutions are slightly more likely to interview or hold 
focus groups with alumni (mean score of 1.96) than with employers (1.84), but do not make 
extensive use of either method. 



Three additional alternative student assessment measures were listed in our sur\’cy: 
transcript analysis, using external examinations, and surveying or interviewing withdrawing 
students. Of these three, doctoral and research institutions are most likely to survey/interview 
withdrawing students (2.18). They make little use of external examinations such as licensure 
examinations (2.02) and transcript analysis ( 1 .92) as student assessment methods. 

Compared to all institutions participating in our study, doctoral and research institutions 
make slightly greater use of capstone courses as an assessment method. Conversely, they make 
much less use of surveys or interviews with withdrawing students. Other alternative assessment 
methods are not used to any greater or lesser extent by doctoral and research universities compared 
to all institutions. 

Assessment Methods for Student Sub-Populations 

Do institutions use different assessment methods for specific sub-populations of their 
student body? Table 4 presents the percentage of doctoral and research universities and all 
institutions using different a.ssessment methods for four student sub-populations: adult students, 
part-time students, minority students, and distance education students. 



Table 4. Institutional Use 
Populations 


of Different Assessment Methods 


for Different Student Sub- 




% Institutions Usinii 


DilTerent Assessment Methods 


Student Sub-Population 


Doctoral and Research (n=143) 


All Institutions (n=1366) 


Adult students 


6 


10 


Part-time students 


5 


5 


Minority students 


4 ‘ 


"> 


Distance education students 


21 


22 



Mo.st doctoral and research institutions do not use different assessment methods for specific 
sub-populations of undergraduate students. While one-fifth (21%) report using different methods 
for distance education students, a decision that may be necessitated by the type of medium used for 
course delivery, very few use special methods for collecting assessment data from their adult, part- 
time or minority students. This profile is consistent across all types of postsecondary in.stitutions. 

Student Assessment Studies and Reports 

Beyond collecting descriptive data on their students' characteristics, performance or 
development, institutions must consider how to transform that data into useful information. 
Institutions that study the relationship between various aspects of students’ institutional 
experiences and their performance will be better able to make informed decisions concerning 
academic and student-related policies and practices. Another consideration in data analysis is the 
level of aggregation at which assessment information is analyzed and reported. 

Student Assessment Studies 

Are institutions studying how student performance is connected to experiences with the 
institution? In our survey, we asked if institutions conduct studies of the relationship between 
students’ performance and nine areas of students’ institutional experiences. Table 5 identifies these 
areas and presents data on institutional studies for both doctoral and research institutions and all 
institutions in our study. 



Table 5. Institutions Conducting Student Assessment Studies 



Studies of Relationship Between Student Perfomiance 
and the Following Experiences 


9^ Institutions 


Conducting Studies 


Doctoral and Re.'icarch 
(n=l.'^9) 


All Institutions (n=1264) 


Do not study any of these relationships 


22 


38 


Admission standards or policies 


60 


42 


Residence arrangements 


48 


21 


Student financial aid and/or concurrent employment 


44 


30 


Student course-taking patterns 


38 


26 


Extra-curricular activities 


37 


24 


Academic advising patterns 


26 


26 


Exposure to different instructional or teaching methods 


25 


21 


Patterns of student-faculty interaction 


25 


21 


Classroom, library and/or computing resources 


12 


17 



The majority of doctoral and research institutions conduct studies of the relationship 
between some aspect(s) of students’ institutional experiences and their perfomiance: less than one- 
quarter of doctoral and research institutions (22%) do not conduct any of such studies. When 
studies are conducted by doctoral and research universities, they are most likely to examine the 
relationship of admissions policies (60%). residence arrangements (48%). financial aid and 
employment (44%), course-taking patterns (38%) and extra-cumcular involvement (37%) to 
.students’ performance. 

However, they are less likely to study the relationship between students' performance and 
aspects of students’ institutional experiences that are more directly related to the teaching/leaming 
environment such as classroom, library and computing resources (12%). interaction with faculty 
(25%), exposure to various teaching methods (25%) and academic advising (26%). Arguably, 
studies of these relationships may produce infomiation that could have a greater impact on student 
performance but it is also more politically sensitive for institutions to conduct them. Or it may be 
that studies of this nature are primarily conducted by faculty members within their own classrooms 
rather than by a central committee or institutional office; such .studies would not necessarily have 
been reported in this institutional-level instrument. 

Compared to all institutions participating in our survey, doctoral and research universities 
are more likely to study the relationship of some aspect(s) of students' institutional experiences and 
performance. Doctoral and research universities are more likely to conduct six of the nine types of 
studies listed, with studies of academic advising, teaching methods, and academic re.sources being 
the exception. These results may reflect both the recent emphasis on undergraduate education 
underway in these institutions and their well-developed capacity for institutional or educational 
research. 

Student Assessment Profiles or Reports 

Finally, institutions were asked to report the levels of aggregation at which they provide 
profiles or reports of student assessment information. Figure 15 shows the percentage of doctoral 
and research universities and all institutions providing student assessment reports at each of five 
levels of aggregation: institution-wide; schools or colleges; academic programs or departments; 
special populations of students; course or groups of courses. Respondents indicated as many levels 
of aggregation as were applicable or could indicate that they provide no reports of student 
assessment information. 



Figure 15. Preparation of Student Performance Reports by Level of Aggregation 
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Most doctoral and research institutions provide some types of reports of student assessment 
results (91%). The majority provide such reports on data for the institution as a whole (73%) and 
for schools or colleges (73%). Slightly fewer prepare reports aggregated at the level of academic 
programs or departments (68%) and for specific sub-populations of students (59%). They are least 
likely to aggregate and report assessment information at the course level (32%). 

Compared to all institutions, doctoral and re.search universities are much more likely to 
provide reports at the school or college level. This reflects the decentralized organization of this 
institutional type. It suggests that schools or colleges in these institutions more often have the 
responsibility to decide what assessment results mean and what to do with them than is the case for 
all postsecondary institutions. To a lesser degree, doctoral and research institutions are also more 
likely to provide reports on specific student sub-populations. 

Summary Observations 

With the exception of basic college-readiness skills, doctoral and research institutions tend 
to collect more extensive data on affective or behavioral measures than on cognitive measures, and 
from current rather than former students. This data collection profile may be partly attributable to 
the ea.se or difficulty of collecting various types of data. Measures of academic progress, academic 
intentions and basic skills ai'e easier for institutions to define, quantify and collect than more 
complex mea.sures such as higher-order cognitive skills and civic or social roles. Although 
differences are generally not large, the overall profile of data collection within doctoral and research 
institutions is slightly less extensive than the profile reported by respondents from all types of 
postsecondary institutions. Prior research suggests this may stem from a belief that the academic 
caliber of students at these institutions obviates the need for assessment (Jemmott & Morante, 
1993). 



Doctoral and research universities frequently use tests or instruments to collect assessment 
data. These instruments are most often developed by institutions themselves and are rarely 
provided by state-level officials. Overall, they make limited use of non-traditional assessment 
methods such as portfolios and capstone courses. 

Compared to all institutions, doctoral and research institutions have a strong profile of 
analyzing student assessment data. However, studies are more likely to examine the influence of 
students’ non-instructional experiences within the institution such as admissions or financial aid 




policies on student performance. Studies of the relationship of students' instructional or 
advisement experiences to their performance are conducted comparatively less often. The majority 
of doctoral and research institutions provide descriptive reports of student perfomiance — most 
often aggregated at the institution-wide and school or college level. 



5. ORGANIZATIONAL AND ADMINISTRATIVE SUPPORT PATTERNS 

Focuses oil the patterns of organizatiomil and administrative support that 
institutions have developed to promote student assessment on their campuses, 
such as their mission and purpose, institution-wide activities, leadership 
support, and planning and coordinating mechanisms. Many doctoral and 
research institutions have adopted a decentralized approach to asses.sment 
policies and planning — pennitting academic units to develop their own 
assessment activities. Executive responsibility for planning and overseeing 
daily operations is primarily positioned in academic affairs. Faculty and 
personnel with research and evaluation expertise are involved hut to a le.s.ser 
extent. 

An important focus of our study was to examine the patterns of organizational and 
administrative support institutions have developed to promote student assessment on their 
campuses. The as,sessment literature suggests that institutions will be most likely to promote 
internal support for assessment if they engage in assessment for internal rather than external 
purposes, have visible and strong leadership support for assessment, include a broad range of 
internal participants, particularly faculty, in making assessment-related decisions and regularly 
evaluate their assessment programs. It is expected that organizational and administrative support 
will shape the assessment approach. as,sessment management practices and policies, and 
ultimately, the institutional uses and impacts of assessment information. 

The ISSA instrument included questions concerning the institution-wide assessment 
support strategy; administrative and governance activities used to promote student assessment; 
leadership and faculty support for assessment; planning and coordination for assessment; and 
evaluation of student assessment processes. Once again, we will present information concerning 
the pattern of organizational and administrative support for student assessment in doctoral and 
research institutions and compare it to the pattern for all types of postsecondary institutions. 

Institution-Wide Assessment Support Strategy 



Mission Emphasis 

Assessment scholars and practitioners suggest that institutions’ academic mission content 
may symbolize the importance with which .student assessment is regarded and so may influence the 
degree of internal support for assessment. What do the mission statements ot our responding 
institutions tell us about their values regarding student assessment? Institutions reported whether 
their mission statement explicitly: a) emphasizes excellence in undergraduate education, b) 
identifies intended student outcornes, c) refers to student assessment as an important priority, or d) 
does not mention any of the.se. Respondents could answer “yes" to more than one mission 
statement component. Figure 16 displays the percentage of doctoral and re.search institutions and 
all institutions reporting each of these mission statement components. 

Despite their dual emphasis on graduate and undergraduate education, the majority of 
doctoral and research universities report their mission statement explicitly emphasizes excellence in 
undergraduate education as an institutional priority (86%). However, just over one-third have 
mission statements that identify the educational outcomes intended for their .students (37%) while 
only 14 percent refer to student assessment as an important institutional activity within their 
mission statement. Few doctoral and research institutions have none of these components in their 
mission statement (1 1%). Compared to all in.stitutions, doctoral and research institutions are much 
less likely to include intended educational outcomes for students in their mi.ssion .statements. 



Figure 16. Institutional Mission Emphasis 
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Purposes for Engaging in Student Assessment 

Why are institutions engaging in student assessment? A second dimension of institution- 
wide assessment support strategy examined in this study is the purpose(s) underlying institutions' 
student assessment efforts. Scholars contend that whether assessment is primarily engaged in for 
internal or external purposes may influence the nature of an institution's assessment approach, 
degree of internal support, and assessment uses and impacts. Institutions rated the importance of 
six purposes for engaging in student assessment: preparing for an institutional accreditation self- 
study; meeting state reporting requirements; guiding internal resource allocation decisions: guiding 
undergraduate academic program improvement; improving the achievement of undergraduate 
students; and improving faculty instaictional perfonnance. The last four of these purposes were 
factored together to create an "‘internal puiposes" index that we used in regression analyses in 
section eight. Figure 17 presents the mean scores for each student assessment purpose for doctoral 
and research institutions and for all institutions. 

Figure 17. Purposes of Student Assessment 
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Scale: 1 = no importance; 2 = minor importance; 3 = moderate importance 4 = very important. 



Doctoral and research colleges and universities rate preparing for an institutional 
accreditation self-study (3.39), guiding undergraduate academic program improvement (3.28), and 
improving the achievement of undergraduate students (3.27) as very important rea.sons for 




engaging in student assessment. Conversely, they give the lowest imponance ratings to guiding 
internal resource allocation decisions (2.45). meeting state reponing requirements (2.50) and 
improving faculty instructional performance (2.71) as student assessment purposes. These 
purposes are rated as being of minor to moderate importance. 

These findings suggest regional accrediting associations play an important role in 
stimulating doctoral and research^^institutions' decisions to engage in student assessment. Similarly, 
they intend their student assessment efforts to contribute to improvements in academic program and 
student performance. It appears that student assessment efforts in doctoral and research institutions 
are more often intended for improvement purposes than to address issues of external or internal 
accountability. 

However, compared to all institutions in our study, doctoral and research institutions 
accord much less importance to accreditation preparation, meeting state reporting requirements and 
improving faculty instruction as assessment purposes. The low importance of the first two 
purposes suggests these external constituencies have less influence over student assessment efforts 
in doctoral and re.search universities than is the case in other types of institutions. The relatively 
low importance rating given to improving faculty instruction may be attributable to the high degree 
of faculty autonomy in these institutions and the greater emphasis placed on research rather than 
teaching as a faculty role responsibility. 

Institution-Wide Administrative and Governance Activities 

Institutions may develop a variety of admini.strative activities and governance structures to 
promote student assessment on an institution-wide basis. How prevalent are these practices? The 
ISSA instrument asked whether or not institutions use any of the following administrative activities 
and governance structures: annual institution-wide initiatives on assessment; rewards or incentives 
for academic and student affairs administrators to use student assessment in their unit; incentives 
for academic units to use assessment information in evaluation and improvement el torts; student 
assessment workshops for academic, and student affairs administrators; board of trtistees committee 
that addres.ses student assessment; faculty governance committee that addresses student assessment 
issues; and student representation on student assessment committees. Figure 18 di.splays the 
percentage of doctoral and research institutions and all institutions that have introduced these 
activities or structures in their institutions. 

Figure 18. Institution-Wide Administrative and Governance Activities Used to Promote 
Student Assessment 
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Of the various activities listed, doctoral and research institutions are most likely to provide 
workshops on student assessment for their academic and student affairs administrators (5990 and 
to have a faculty governance committee that addresses student assessment issues (469f ). A smaller 
proportion offer incentives for academic units to use assessment information in improvement 
efforts (34%), include student representation on student assessment committees (32%) and have 
annual institution-wide initiatives on assessment (31%). Doctoral and research institutions are less 
likely to offer rewards or incentives to individual administrators who promote student assessment 
(18%) and least likely to have a board of trustees committee focused on student assessment (10%). 

Relative to all institutions, doctoral and research universities are more likely to provide 
incentives to encourage administrators and academic units to use student assessment intomiation. 
They are less likely to have institution-wide initiatives or forums on student assessment and to 
involve faculty in assessment governance. This suggests that student assessment is primarily an 
administrative responsibility rather than a faculty responsibility within these institutions. 

Leadership and Faculty Support for Student Assessment 

The degree to which student assessment efforts are supported internally is likely an 
important determinant of the extent and impact of an institution's student assessment activities. 
How supportive of student assessment are various internal constituencies? We asked survey 
respondents to rate the degree to which six internal groups support undergraduate student 
as.sessment activities: board of trustees; chief executive officer; academic affairs administrators; 
student affairs administrators; faculty governance; and students. Figure 19 presents the mean 
responses to this question for doctoral and research universities and all institutions in our study. 

Figure 19. Internal Constituent Support for Student Assessment 




Scale: 1 = very unsupportive; 2 = somewhat unsupportive; .1 = neutral, unknown; 4 = somewhat supportive; 
5 = very supportive 



Doctoral and research universities have generally high levels of support for student 
assessment with most constituents reported as being at least somewhat supportive. Academic 
affairs administrators are perceived as the most supportive constituency (4.43) followed by student 
affairs administrators (4.29) and the chief executive officer (4.22). Boards of trustees (3.68) and 
faculty governance bodies (3.51) are viewed as slightly less supportive in relation to senior 
administrators but are still rated as supportive. Students are seen as being least supportive of 
student assessment with most respondents reporting their level of support as neutral or unknown 
(3.29). This latter finding may warrant further attention given scholars’ concerns about the need to 




actively promote student participation in assessment activities. In all cases, the levels ot internal 
constituent support for assessment reponed by doctoral and research universities are equivalent to 
those reported by all institutions in our study. 

Planning and Coordinating Student Assessment 

A central issue in discussions of leadership and governance for student assessment 
concerns the degree to which responsibility for planning and coordinating student as.sessment 
activities is centralized or decentralized within institutions. Centralized approaches that locus such 
responsibility on senior administrators or central offices may signal that assessment is a valued 
institutional activity but decentralized approaches involving an array of internal participants may do 
more to promote broader internal support for assessment. How do institutions plan and coordinate 
their assessment efforts? We asked institutional respondents about six aspects of their assessment 
planning and coordination process: the nature of an institutional plan or policy for student 
as,sessment; membership on an institution-wide planning group for assessment; executive 
responsibility for the assessment planning process; approval authority for assessment plans; 
operational responsibility for student as.sessment; and reporting patterns. 

Institutional Plan or Policy for Student Assessment 

We asked survey respondents which of .seven type.s of institutional plans or policies for 
student as.sessment best described what exists at their institution: 1) fomial comprehensive 
centralization — a formally adopted plan or policy specifying undergraduate student as.sessment 
activities for all academic programs or units; 2) formal limited centralization — a fomially adopted 
plan or policy for undergraduate student assessment in some academic programs or units; 3) formal 
decentralization — a formally adopted institutional plan or policy requiring all academic units or 
programs to develop their own undergraduate student as.sessment plan; 4) formal guidance — a 
formally adopted institutional plan or policy identifying institution-wide activities to be conducted 
by a central committee or office; 5) informal encouragement — no institutional plan or policy but 
academic units or programs are encouraged to develop their own undergraduate student as.sessment 
activities; 6) emergent — institution's currently developing a plan or policy for undergraduate 
student asse.ssment; and 7) none — institution does not have an undergraduate student as.sessment 
plan or policy. Respondents could select more than one type of plan or policy. Figure 20 prc.sents 
the percentage of doctoral and research institutions and all institutions in our study using each type 
of assessment plan or policy. 

Most doctoral and research institutions have some type of institutional plan or policy for 
undergraduate student assessment. Half (49%) report having a policy that requires all academic 
units or programs to develop their own undergraduate student as.sessment plan. Approximately 
one-third (35%) .stipulate in.stitution-wide as.sessment activities that are conducted by a central 
committee or position. About the same proportion (32%) require specific as.sessment activities of 
all their academic units or program. They make limited u.se of the other types of policies and plans 
considered in this question. 

Compared to all institutions in our study, doctoral and re.search universities are somewhat 
less likely to have a formal plan or policy for student assessment. If one combines informal 
encouragement, emergent plans and no plans, fully 41% of doctoral and re.search universities have 
no formal assessment plan or policy. They are more likely than all institutions in our study to adopt 
a decentralized assessment plan or policy in which academic units and programs determine the 
assessment activities in which they will engage, and less likely to use a formal centralized policy 
that mandates particular assessment activities of all units or programs. This''approach to 
institutional policies or plans for student assessment is consistent with the overall governance style 
in these larger, organizationally complex and decentralized institutions. 



Figure 20. Types of Institutional Plan or Policy for Student Assessment 
100 - 



O) 

c 

w 

D 

w 

c 

o 



w 

c 



90- 


1 ^ 1 


■ Doc/Res 


t 

1 


80- 


1 


i 

1 


70- 


1 


□ A 1 1 


1 

( 

1 




cent ral 

comprehensive 



cent ral 
limit ed 



formal 

decentralized 



formal 

guidance 



informal 

encouraged 



emerging 

plans 



none 



Institulion-Wide Assessment Planning Group Membership 

Who is involved in student assessment planning? We asked respondents whether they have 
an institution-wide group responsible for ongoing planning and policy setting for undergraduate 
assessment. Only 53% of doctoral and research universities report having such a group compared 
to 70% of all institutions in our study. Of those institutions with institution-wide planning groups, 
we asked which of the following internal constituents serves on the group; chief executive officer, 
academic affairs administrator or staff, student affairs administrator or staff, institutional research 
administrator or staff, academic review and evaluation administrator or staff, student assessment 
administrator or staff, faculty, and students. Respondents checked as many constituents as were 
applicable. Figure 21 presents the percentage of doctoral and research institutions and all 
institutions that responded that each constituent is a member of their assessment planning group. 



Figure 21. Membership on Institution-Wide Student Assessment Planning Group* 
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*Only institutions, with an institution-wide planning group for student asse.ssment responded to this question. 




When doctoral and research universities have institution-wide planning committees for 
student assessment, these committees are predominantly comprised of faculty (89%) and academic- 
affairs administrators (85%), institutional research personnel (61%) and student affairs 

• administrators (50%). Student assessment administrators (43%). students (36%) and personnel 
with specific responsibilities for academic review/evaluation (32%) are represented to a lesser 
degree. It is least likely for the chief executive officer (3%) to be included. 

Compared to all institutions in the study, doctoral and re.search universities are more likely 
to include personnel with specific responsibility for student assessment on this committee and less 

• likely to involve the chief executive officer. Student assessment planning is primarily positioned as 
an academic affairs responsibility in doctoral and re.search institutions. Staff with specialized 
research and assessment expertise are also well repre.sented. 



Executive Responsibility for Assessment Planning Process 

Another aspect of the governance process used for student assessment concerns where 
formal leadership for assessment planning is vested in an institution. We asked institutions which’ 
of the following positions has executive responsibility for or chairs the institution-wide group for 
student assessment planning: academic affairs administrator; student affairs administrator: 
institutional research officer; academic review and evaluation officer: student assessment officer; or 
faculty member. Figure 22 presents this information for doctoral and research institutions and all 
institutions. 



Figure 22. Executive Responsibility for Student Assessment Planning Group* 
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*Only inslilulions with an in.slilulion-wide planning group for .student assessment responded to this question. 



Executive responsibility for student assessment planning in doctoral and research 
institutions is most often placed in the hands of an academic affairs administrator (54%) followed 
by a faculty member (28%). It is less common for personnel from institutional research (14%) or 
0 academic review and evaluation (14%) to hold this position. Doctoral and research institutions 

rarely invest executive responsibility for assessment planning in a student affairs administrator or 
student assessment officer (4% each). There are few differences noted between this profile of 
executive responsibility for doctoral and research universities and all institutions. 



Approval Authority for Student Assessment Plan 
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As a final aspect of governance for assessment planning, we asked institutions to identify 
who, among ten possible positions or groups, approves changes in their institutional plan or policy 

• for student assessment: board of trustees; chief executive officer; chief academic affairs officer; 
chief student affairs officer; institutional research officer; academic review and evaluation officer; 
student assessment officer; student government; academic senate or other faculty committee; and 
faculty union. All respondents could answer this question regardless of whether or not they have 
an institution-wide committee for assessment planning. Respondents indicated as many positions 
or groups as were applicable. Less than 10 percent of doctoral and research universities report that 

• board of tmstees, .student assessment, student government and faculty union personnel have 
approval authority for assessment planning. Figure 23 presents the results for the remaining six 
positions for doctoral and research universities and all institutions. 



Figure 23. Approval Authority for Student Assessment Plan or Policies 
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9 Changes to student assessment plans or policies have to be approved by multiple sources. 

However, doctoral and research universities most often give approval authority to the chief 
academic affairs officer (82%). In terms of frequency, the chief executive officer and academic 
senate are the internal constituencies mentioned next (26% each) followed by the chief executive 
officer (37%). There is less involvement of other administrative or research/evaluation positions in 
this aspect of assessment governance. Compared to all institutions, doctoral and research 
9 universities are less likely to include the chief executive officer and the academic senate in 

approving changes to the institutional assessment plan or policy. 

Operational Responsibility for Day-to-Day Student Assessment Activities 

Beyond the planning phase of student assessment, institutions must decide where to place 
9 responsibility for overseeing their day-to-day undergraduate student assessment activities such as 

instmment development, data collection, analysis and reporting. Where have institutions placed 
operational responsibility for student assessment? We asked institutions which of six positions or 
offices has responsibility for their day-to-day student assessment activities: academic affairs 
administrator; student affairs administrator; institutional research officer; academic review and 
evaluation officer; student assessment officer; and faculty member(s). Institutions checked as many 
9 positions/offices as applied. Figure 24 presents the results for doctoral and research institutions 

and all institutions. 
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Figure 24. Operating Responsibility for Day-to-Day Student Assessment Activities 



05 

c 



o 

Q. 

05 

CC 





academic student institutional academic student faculty 

affairs affairs research review assessment member 



Doctoral and research universities are equally likely to place responsibility for their day-to- 
day student assessment activities with an academic affairs administrator (44%) or institutional 
research personnel (43%). This responsibility is less frequently given to faculty members (23%). 
Conversely, doctoral and research universities seldom place a student attairs administrator (19%). 
academic review/evaluation officer (16%) or student assessment officer (14%) in charge of its day- 
to-day as.sessment activities. There are no major differences in the assignment of operational 
responsibility for student assessment within doctoral and re.search universities and all institutions. 

Reporting Relationship of Individual with Operating Responsibility for Student Assessment 



Finally, institutions were asked to whom the individual with day-to-day operating 
responsibility for student assessment reports. Survey respondents selected from the following five 
offices: chief executive officer; chief academic officer; chief student affairs officer, institutional 
research officer; and academic review and evaluation officer. Figure 23 presents the resj^onses 
® from doctoral and research institutions and all institutions in our study. 



In the majority of doctoral and research institutions (68%), the individual with operational 
responsibility for student assessment reports to the chief academic officer. Less than 10 percent of 
doctoral and research institutions have the operational manager for student assessment report to any 
of the other administrative or research/evaluation positions listed. Compared to all institutions in 
® our study, doctoral and research universities are more likely to have the operational manager for 

as.sessment report to the chief academic officer and less often, to the chief executive officer. 

Evaluating the Student Assessment Process 



The student assessment literature insists upon the importance of institutions regularly 
® evaluating their assessment processes. Consequently, our survey inquired whether institutions 

have evaluated their student assessment plan or process and if .so, what elements of their plan or 
process have been reviewed. 
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Figure 25. Reporting Relationship of Individual with Operating Responsibility for Student 
Assessment Activities 
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Status of Evaluation of Student Assessment Plan or Process 



Institutions were asked whether they have conducted a formal evaluation, an informal 
evaluation, are cun*ently developing a plan for an evaluation, or are not planning to evaluate their 
assessment process. The results for doctoral and research institutions and all institutions are 
depicted in Figure 26. 
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Figure 26. Status of Student Assessment Evaluation 
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Doctoral and research institutions who have evaluated their assessment process have more 
often done so informally (26%) than formally (17%). One-quarter are developing evaluation plans 
(24%). However, doctoral and research universities are more likely than all institutions to be 
neither evaluating nor planning to evaluate their assessment process (33% of doctoral/research 
institutions versus 21% of all institutions). The low incidence of actual evaluation and evaluation 
® planning may be partly attributable to the newness of assessment activities among all 
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postsecondary institutions. It does mean that the majority of doctoral and research universities do 
not know how well their assessment processes are functioning. 

Elements of Student Assessment Evaluation 

Those institutions that had formally or informally evaluated their student as.sessment 
process were asked which of eight elements were reviewed as part of the evaluation: student 
a.ssessment plans and policies; structure and responsibility for student as.sessment; achievement of 
intended objectives for student assessment; reliability and validity of assessment instruments and 
methods; quality of data analysis; use of assessment information in institutional decision-making; 
problems encountered while conducting asse.ssment; and the costs and benefits of student 
assessment. Results of this question for doctoral and research institutions and all institutions are 
presented in Figure 27. . 



Figure 27. Elements of Student Assessment Evaluation 




When evaluating their student assessment process, doctoral and research institutions most 
often focus on the a.ssessment plan itself (73%). A large proportion consider administrative or 
managerial elements such as problems encountered while conducting assessment (66%), the use of 
assessment information in institutional decision making (59%), the structure and responsibility for 
as.sessment (56%), the achievement of intended objectives (56%), as well as technical aspects of 
as.sessment such as instrument reliability and validity (42%) and data analysis quality (44%). 
Institutions rarely evaluate the comparative costs and benefits of student assessment (5%). 
Compared to all institutions, doctoral and research institutions are less likely to evaluate whether 
the as.sessment process has achieved its intended objectives, assessment instruments or methods, 
and costs and benefits of assessment. 

Summary Observations 

The preceding sections have examined various aspects of doctoral and research institutions’ 
organizational and administrative support for student as.sessment. The overall pattern reveals a 
student assessment support strategy that is more internally than externally focused in its orientation 
and a leadership and governance approach that is quite decentralized in nature. Evaluation efforts 
appear to be still emerging. 

The mission statements of doctoral and research universities prioritize undergraduate 
education although few make explicit mention of assessing student performance or intended 



educational outcomes. With the exception of improving faculty instruction, survey respon.ses from 
doctoral and research institutions suggest their student assessment efforts are more often intended 
to address internal than external purposes. Although preparing for an institutional accreditation 
self-study is rated as a very important assessment purpose, doctoral and research universities 
accord less importance to accreditation and state requirements as as.sessment purposes than do all 
institutions. 

Considering their use of administrative and governance activities, doctoral and re.search 
universities rely on professional development opportunities and. to a les.ser extent, offer incentives 
or rewards as a means of encouraging administrative involvement in student as.sessment. Faculty 
and student involvement is promoted through inclusion in governance structures for asse.ssment, 
although the former occurs to a lesser degree in these institutions than in all institutions. Internal 
leadership support for student assessment is generally perceived as high. Senior administrators are 
viewed as the internal constituency most supportive of student assessment, followed by trustees 
and faculty. 

Compared to all institutions in our study, doctoral and re.search institutions are more likely 
to take a decentralized approach to assessment plans and policies. This is reflected in the form of 
institutional policy for student assessment that typically exists — requiring some form of student 
asse.ssment activity of all academic units but permitting units to develop their own as.sessment plans 
and activities. Planning and coordinating responsibility for student as.sessment is chiefly situated in 
academic affairs rather than in the chief executive or student affairs offices. Faculty are moderately 
well represented in all aspects of the student as.sessment planning process — most often .serving on 
an institution-wide planning group, but less often holding executive responsibility for this group or 
approving changes in the assessment plan or policy. Oversight of day-to-day student assessment 
activities is most often assigned to an academic affairs administrator or institutional re.search 
personnel, with faculty members less frequently given this responsibility. The individual with 
operational responsibility most often reports to the chief academic officer. 

Evaluation of the student assessment plan and process is not yet a well-developed practice 
in postsecondary institutions. This is particularly so for doctoral and re.search universities. Fully 
one-third of these institutions have neither evaluated nor are planning to evaluate their assessment 
process. However, most elements of a good evaluation are considered by doctoral and re.search 
universities that have formally or informally evaluated their student as.sessment process. 



6. ASSESSMENT MANAGEMENT PRACTICES AND POLICIES 

Reports on the existence of specific institutional practices anil policies 
designed to promote student assessment management. Doctoral and research 
institutions prefer unobtrusive practices and policies such as distributing 
assessment results and providing access to .student petfonnance information, 
to more obtrusive practices and policies such as using student assessment 
infonnation in making either budget allocation or facnits- evaluation and 
reward decisions. Compared to all institutions, they make less e.xtensive it.se 
of assessment management practices and policies. 

A fourth domain in our conceptual framework is that of assessment management practices 
and policies that provide mechanisms for managing the student assessment process and direct the 
ways in which student assessment information is used throughout the institution. Assessment 
management practices and policies are suggested in the literature as powerful means through which 
institutions can support student assessment and encourage the use of collected assessment 
information. Conceptual dimensions of asse.ssment management practices and policies such as their 
comprehensiveness, consistency, and the extent to which they are employed within an institution 
are expected to influence internal support for student assessment and the likelihood of achieving 
institutional impacts from as.sessment information. 

Our survey asked about specific institutional practices and policies promoting student 
assessment management. These items were factor analyzed to create nine comprehensive indices. 

In this chapter we will present results of these assessment management practice and policy indices 
rather than of individual items. 

Practices for Managing Student Assessment 

What kinds of practices do in.stitutions develop to manage and promote their student 
assessment process? We asked respondents about the existence of a variety of specific institutional 
practices used to manage the student assessment process in four areas: academic resource 
allocation, student information systems, internal access to student assessment inlormation. and 
distribution of assessment reports and .studies. Scores for institutions for all specific practices in 
these four sections are presented in Appendix A. We used factor analysis to reduce these responses 
to indices of management practices in these four areas. Detailed information on these indices is 
provided in Appendix B. Briefly, the four indices are: 

1 . Academic Budget Decisions: whether the institution's academic budget process compares 
academic units on student performance indicators and allocates resources competitively among 
them; or rewards them for improvement based on student performance indicators. 

2. Computer Support: whether the institution has a computerized student information system 
which includes student performance indicators; tracks students from application through 
graduation; and is integrated with faculty, curricular and financial databases. 

3. Access to Student Assessment Information: whether assessment information for individual 
students is available to institutional research, assessment or evaluation professionals; senior 
academic administrators; department chairs or academic program administrators; student affairs 
professionals; and faculty advisors. 



4. Distribution of Assessment Repons; whether assessment reports are regularly distributed to 
students, faculty, academic administrators, student affairs professionals, employers, and the 
general public. 

Figure 28 presents mean scores for each of these four indices. Each was originally- 
measured on a different scale. For the purposes of comparison, we have translated each index to a 
scale of zero to five. Therefore, we can determine which of these practices is more prevalent in 
doctoral and research institutions and all institutions in our study. Frequencies for specific items 
included in each index are taken from Appendix A. 

Figure 28. Extent of Institutional Use of Assessment Management Practices 
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Scale: 0 = institution used no practices in index; .S = institution used all practices in index 



Of these four indices of assessment management practices, doctoral and research 
in.stitutions are most likely to provide access to assessment information to a number of internal 
constituents (mean index score of 2.86). For example, one of the items comprising this index is 
whether this information is accessible to institutional researchers: 69% ot doctoral and research 
institutions in our study report that it is. Moderately high proportions of doctoral and research 
universities (55% to 62%) report that student as.sessment information is available to a variety of 
academic personnel. There is one exception to this pattern of internal access: less than half (44%) 
of doctoral and research institutions grant such information access to student affairs professionals. 
Although providing internal access to student assessment information is the most prevalent of the 
four management practices reported by doctoral and research institutions, they are less likely to 
provide such access than are all institutions in our study. 

The next most frequent student assessment management practice is distributing student 
assessment reports to six internal and external constituents (mean index score of 1.9). Doctoral and 
research institutions are most likely to regularly distribute assessment reports to academic 
administrators (85%) and, to a lesser extent, student affairs professionals (57%) and faculty 
(50%). Very few regularly distribute such reports to students (18%), employers (6%) or the 
general public (13%). Compared to all institutions, doctoral and re.search universities are less likely 
to distribute assessment reports to faculty (50% of doctoral/research universities versus 67% of all 
institutions). 

In comparison to the above practices, doctoral and research institutions have limited 
information system capabilities for student assessment (mean index score of 1.37). One of the 
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items in this index is whether the institution has a student information system that tracks students 
from application through to graduation: only half (50%) of doctoral and* research institutions in our 
study have such a system. Even fewer (27%) have a computerized information system which 
includes student performance indicators and almost no doctoral and research universities (6%') have 
a student assessment information system that is integrated with other institutional databases. 
Doctoral and research institutions do not report significantly different information system 
capabilities than all institutions in our study. 

Using student assessment information in the budget process to compare and reward 
academic units is almost a non-existent practice (mean index score of .29). Virtually no doctoral 
and research universities use student performance indicators as the basis tor competitively 
allocating -resources among academic units (1%) or rewarding academic units for perfomiance 
improvement (5%). Doctoral and research institutions do not differ appreciably from all institutions 
in their use of these resource allocation practices. 

Policies Supporting and Promoting the Use of Student Assessment 

While the previous section focused on practices used to manage student assessment, this 
section focuses on the policies used to both support student assessment and guide how the 
resulting data are incorporated into other institutional processes. We asked survey respondents 
about the extent to which they have developed a variety of specific institutional policies to support 
and promote student assessment in five areas: student involvement in assessment; professional 
development for faculty and academic affairs administrators; training for student affairs personnel; 
faculty evaluation and rewards; and academic planning and review processes. Scores for 
institutions for all specific policies are presented in Appendix A. Factor analysis produced five 
indices of student assessment policies. Detailed information on these indices is provided in 
Appendix B. Briefly, these five indices are: 

1 . Student Involvement: extent to which the institution requires students to participate in student 
assessment activities, provides students with information on the purpose and uses of student 
assessment, and provides individual feedback regarding student performance results. 

2. Professional Development: extent to which the institution provides funds for faculty to attend 
asse.ssment conferences, faculty workshops or consultative services on student assessment, 
assistance to faculty to improve use of student assessment, and workshops/seminars for 
academic administrators on assessment. 

3. Student Affairs Training: extent to which the institution requires student affairs staff to receive 
training on assessment and provides student assessment workshops tor student affairs 
administrators. 

4. Faculty Evaluation: extent to which the institution considers evidence of student perfonnance in 
faculty evaluation for promotion; incorporates evidence of student performance into faculty 
evaluation for salary and merit; considers faculty scholarship on assessment in promotion, 
tenure or salary reviews; considers faculty participation in assessment in promotion, tenure or 
salary reviews; and recognizes faculty for effective use of assessment. 

5. Academic Planning and Review: extent to which the institution incorporates student 
performance data into academic department or undergraduate program planning or review; 
general education or core curriculum review; course-level review and development; and review 
and planning for student academic support .services. 

These indices summarize the extent to which these policies exist at institutions based on a 
five point scale: 1 = not done at all; 2 = done in a few departments; 3 = done in some departments; 
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4 = done in many depanments; 5 = done in most departments. Figure 29 presents the mean scores 
for these five indices for doctoral and research institutions and all institutions in our study. Mean 
scores for items within each index are taken from Appendix A. 



Figure 29. Extent of Institutional Use of Assessment .Management Policies 
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l=not done at all; 2=done in a few depls; .^=done in some depls; 4=done in many dcpls; 5=donc in most dcpis. 



Of the five indices of student assessment policies presented in Figure 29, doctoral and 
re.search institutions are most likely to incorporate student assessment data into academic planning 
and review processes (mean index score of 2.45). They make fairly extensive use of student 
a.ssessment data in planning and review processes for academic departments or undergraduate 
programs (mean item score of 3.33) and general education or core curriculum (mean item score of 
3.10) — doing so in some to many departments. They make slightly less use of this information in 
proces.ses for reviewing and planning courses (mean item score of 2.92) and .student academic 
support services (mean item score of 2.77). Compai'ed to all institutions in our study, doctoral and 
research institutions are not as likely to use assessment information in the academic planning and 
review processes. 

Doctoral and research institutions make moderate use of policies encouraging student 
involvement in assessment activities (mean index score of 2.26). Institutional scores for specific 
policies comprising this index reveal that, on average, some to many departments require .students 
to participate in assessment activities (mean item score of 3.15) and provide students with 
information regarding the purposes and uses of student assessment (mean item score of 3. 1 1 ). A 
few to some departments provide students with individual feedback regai'ding their assessment 
results (mean item score of 2.62) but only a few departments provide incentives as a means of 
encouraging student involvement in assessment activities (mean item score of 1.85). Doctoral and 
research institutions make less extensive u.se of policies promoting student involvement in 
as.sessment than do all institutions in our study. 

Doctoral and research institutions are less likely to have policies on professional 
development for student assessment. A few departments provide assessment-related professional 
development for student affairs staff or administrators (mean index score of 1.73). Policies 
regarding professional development for faculty and academic administrators are even le.ss prevalent 
(mean index score of 1.63). A few to some departments offer workshops or seminars on using 
student assessment information to their faculty (mean item score of 2.65) and academic 
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administrators (mean item score of 2.23) or provide funds for faculty to attend assessment 
conferences (mean item score of 2.37). They are less likely to encourage faculty use of assessment 
information by providing assistance such as paid leaves or course reduction (mean item seore of 
1.88) or to require faculfy to receive training on student assessment (1.66). Compared to all 
institutions, doctoral and research institutions make less extensive use of professional development 
policies to support their student assessment efforts. 

Many departments within doctoral and research institutions report that they encourage their 
faculty to assess student learning in classes (mean item score of 3.35). However, respondents 
report that a few departments consider evidence of student performance in decisions concerning 
faculty evaluation for promotion (mean item score of 1 .74) or annual salary and merit increases 
(mean item score of 1.69). They are slightly more likely to consider faculty scholarship on 
assessment (mean item score of 2.10) and participation in assessment (mean item score of 1.79) in 
evaluation decisions. A few departments publicly recognize faculty members who have made 
effective use of student assessment (mean item score of 1 .48) or consider experience in student 
assessment when hiring faculty (mean item score of 1.42). But on the whole, like all institutions in 
our survey, doctoral and research universities do not link faculty evaluation and reward policies to 
student assessment (mean index score of 1.23). 

Summary Observations 

In the previous section we concluded that patterns of organizational and administrative 
support for student assessment are decentralized within doctoral and research institutions, more 
often directed toward addressing internal than external purposes, and that responsibility and 
coordination for student assessment is typically positioned within academic affairs. Findings 
regarding the use of assessment management practices and policies are consistent with these 
patterns. 

Most doctoral and research institutions permit institutional researchers and a variety ot 
academic personnel access to assessment information on individual students. Conceivably, 
providing this information access should assist faculty and academic administrators in advising or 
intervening with students as appropriate. Such access is less often extended to student atlairs 
personnel. A.ssessment information distribution is most often intended for internal rather than 
external purposes, and more often for academic administrators than for other internal 
constituencies. While half of doctoral and research institutions can track their students from entry 
to graduation, comparatively fewer have merged assessment data within their information systems 
or can integrate assessment data with other institutional databases. This may constrain the ability ot 
doctoral and re.search institutions to .systematically analyze student assessment data and, especially, 
to examine how student performance is related to aspects of students' educational experiences. 
There is very little use of student assessment information in determining resource allocations 
among academic units. 

Doctoral and research institutions use student assessment information in academic planning 
and review proces.ses but less often incorporate this information in planning for academic support 
services. Some departments have policies intended to encourage student involvement in assessment 
activities. While many departments report that they encourage their faculty members to assess 
student learning in their classes, few provide professional development support for student 
assessment and even fewer include assessment-related criteria within their faculty evaluation and 
reward policies. 

Like all institutions in our study, doctoral and research universities rely on unobtrusive 
practices and policies to promote student assessment. That is, they are more likely to provide 
access to student assessment information and to use this information in academic planning than 
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they are to adopt more contentious policies such as evaluating and rewarding academic programs or 
faculty members on the basis of assessment-related indicators. 

Compared to all institutions, doctoral and research universities make less extensive use ot 
assessment management practices and policies. This is consistent with our finding in the previous 
section that doctoral and research universities generally adopt a decentralized approach to 
organizing and administering student assessment. While some departments may establish specific 
practices or policies to promote student assessment, it is uncommon for these universities to enact 
such practices and policies throughout the institution. 



7. USES AND IMPACTS OF STUDENT ASSESSMENT 






Explores nvo important questions regarding, how institutions use student 
assessment infonnation in decision-making and the impact it has on 
institutions. Doctoral and research institutions are not using student 
assessment infonnation to any great degree in making academic decisions and 
report that student assessment has had little impact on either their internal 
processes or their e.xtemal relationships. 

As noted in our introduction, effective student assessment processes contribute to 
improvements in institutional and student performance. Consequently, two important questions in 
our research are: 1) how have institutions used student assessment infonnation and 2) what impact 
has student assessment information had on institutions? 

From the literature we identified and focused on two critical dimensions: the use of student 
assessment information in academic decision making, and the internal and external impacts on the 
institution that have resulted from student assessment. Our surv'ey included ten academic decisions 
and fourteen institutional impacts. Survey respondents used a four-point scale to indicate the extent 
to which student assessment information had influenced these academic decisions in the institution 
(1 = no action or influence unknown; 2 = action taken, data not influential; 3 = action taken, data 
somewhat influential; and 4 = action taken, data veiy influential). Survey respondents also used a 
four-point scale to indicate the impact that student assessment information has had on the internal 
and external items indicating institutional perfomiance (1 = not monitored, do not know; 2 = 
monitored, negative impact; 3 = monitored, no known impact; 4 = monitored, positive impact). 
Institutions’ scores for each use and impact item are presented in Appendix A and details of the 
indices are presented in Appendix B. Factor analysis clustered these individual items into five 
indices of assessment uses and impacts, two reflecting assessment information uses in academic 
decisions and three reflecting impacts of student assessment on the institution. Briefly, the five 
indices are: 

1 . Educational Decisions: the extent to which student assessment information has been used to 
revise undergraduate academic mission or goals; design or reorganize academic programs or 
majors; design or reorganize student affairs units; allocate resources to academic units; modify 
student a.ssessment plans, policies, or processes; modify general education curriculum; modify 
student out-of-class learning experiences; create or modify distance learning initiatives; modify 
teaching methods; and modify student academic support services. 

2. Faculty Decisions; the extent to which student assessment infonnation has been used to decide 
faculty promotion and tenure; and faculty salary increases or rewards. 

3. Faculty Impacts: whether student assessment information has stimulated campus discussions of 
undergraduate education; contributed to faculty satisfaction; contributed to faculty interest in 
teaching; and led to changes in teaching methods used. 

4. Student Impacts: whether student assessment information has affected students’ satisfaction; 
retention or graduation rates; grade performance; and achievement on external examinations. 

5. External Impacts: whether student assessment information has affected student application or 
acceptance rates; allocation of state funding; evaluation from regional accreditation agency; 
private fund-raising results; success on grant applications; communications with external 
constituents; and institutional reputation or image. 
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Figure 30 presents the means for these five indices for doctoral and research institutions 
and all institutions. Mean scores for individual items comprising each index are taken from 
Appendix A. 



Figure 30. Institutional Uses and Impacts of Student Assessment Information 
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*Use scale; l=no action or influence unknown. 2=aciion taken, data not influential; .i=action taken, data somewhat 
influential; 4=action taken, data very inlluential. 

**Impact scale: l=not monitored, do not know; 2=monitored. negative impact; .'^=monitorcd. no knttwn impact; 
4=monitored. positive impact. 



Uses in Academic Decisions 

As Figure 30 shows, student assessment information has had limited influence on 
educational decisions (mean index score of 1.30) within doctoral and research institutions and even 
less on faculty-related decisions (mean index score of 1.20). A comparison of institutions' mean 
scores for specific items comprising each of these indices provides more detail about the uses made 
of a.ssessment information. 

Concerning individual items within the educational decisions index, doctoral and research 
universities have most often used student as.sessment infomiation to revise student academic 
support services (mean item score of 2.62) and student assessment plans or processes (mean item 
score of 2.41), to design or reorganize academic programs or majors (mean item score of 2.35) 
and to modify teaching methods (mean item score of 2.34) and general education curriculum (mean 
item .score of 2.30). These mean scores indicate assessment information has been somewhat 
influential in the.se decisions. Student assessment information is least likely to have intliienced 
decisions concerning distance learning initiatives (mean item .score of 1.58), resource allocation to 
academic units (mean item score of 1.62), and the undergraduate academic mission (mean item 
score of 1.70). 

More than two-thirds of respondents from doctoral and research institutions are unaware of 
the influence of assessment information on decisions concerning faculty promotion and tenure 
(72%) and faculty salary increases or rewards (72%), the individual items comprising the faculty 
decisions index. For the most part, doctoral and research institutions report that student assessment 
information has had no influence on decisions regarding faculty promotion and tenure (mean item 
score of 1.34) or faculty salary increases or rewards (mean item score of 1.33). 



Compared to all institutions, doctoral and research institutions report less influence ot 
student assessment information on educational decisions and faculty decisions although differences 
are not large. Overall, assessment information appears to have had limited influence on institutional 
# decision making. 



Institutional Impacts 

Figure 30 also displays the mean score for student assessment impacts on institutions' 
faculty, students and external relationships. Doctoral and re.search institutions are slightly more 
likely to report student assessment information has affected indicators ot faculty (mean index score 
of 1.41) and student (mean index score of 1.51) performance than external indicators of 
institutional performance (mean index score of 1.13). Again, a comparison of institutions' mean 
scores for specific items comprising these three indices provides more detail about the impact of 
student assessment information. 

Within the faculty impacts index, doctoral and research institutions are most likely to 
attribute changes in faculty teaching methods (mean item score of 2.29) and more frequent 
discussions of undergraduate education (mean item score of 2. 13) to student assessment 
information. They have more often documented positive impacts from assessment infomiation on 
student retention or graduation rates (mean score of 2.04) than on student satisfaction (mean item 
.score of 1.93), performance on external examinations (mean item score of 1.85) or grades (mean 
item score of 1 .80). 

Institutions are least likely to report that student assessment has had an impact on external 
measures of institutional performance. With one exception, mean scores of individual items 
comprising this index indicate that most doctoral and research institutions have not monitored 
external impacts of their student assessment efforts (mean item scores range from 1.42 to 1.76). 
The exception to this pattern concerns regional accreditation evaluations: doctoral and research 
universities are more likely to report a positive impact from student assessment information on this 
indicator (mean item score of 2.50) than any other individual measure of external perfomiance 
considered in the survey. 

Overall, as the mean scores presented in Figure 30 clearly show, most institutions 
participating in our study, doctoral and research universities and all institutions alike, have not 
monitored the impact of student asse.ssment on internal and external indicators of institutional 
performance. Doctoral and research institutions do not differ appreciably from all institutions in our 
study in their reports of student assessment impacts. 

Summary Observations 

To date, doctoral and research institutions have made limited use of student assessment 
information for academic decisions. Many respondents are unaware of whether assessment 
information has been influential or not in shaping institutional decisions. When used, assessment 
information is more likely to inform educational decisions concerning academic and assessment 
planning than decisions regarding faculty evaluation and rewards. 

Similarly, very few institutions have monitored the impact of student assessment 
information on faculty and student performance and even fewer have done .so for external . 
indicators of institutional performance. When monitoring has been undertaken, doctoral and 
research institutions most often report assessment information has affected regional accreditation 
evaluations, led to changes in teaching methods used by faculty and stimulated discussions of 
undergraduate education. These findings indicate that student assessment has not yet become 
entrenched in doctoral and research institutions. 
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8. KEY RELATIONSHIPS 



Institutional factors, such as conducting assessment for internal improvement 
purposes, and e.xtemal factors such as .state requirements and regional 
accreditation membership influence the type and e.vtent of .student assessment 
data collected b\ doctoral and research institutions. In.stitution.s are more likely 
to report using student assessment infonnation in academic deci.sions if they 
make extensive use of a variety of assessment management practices and 
policies to promote their assessment efforts, conduct studies of collected 
assessment data and use institution-wide .support strategies. A number of 
institutional choices and activities are also .strong detenninants of achieving 
positive impacts from assessment infonnation. Doctoral and Research 
institutions that analyze collected a.ssessment data, sponsor institution-wide 
administrative and governance activities and use a.s.ses.sment management 
practices and policies to support assessment efforts report greater impacts 
from student assessment on faculty and student peifonnance and e.xtemal 
relationships. Private institutions are less likely to use or be impacted by 
assessment information. E.xtemal forces have little influence on a.sse.ssment 
u.ses and impacts. 

In the previous sections we have examined external inlluences. institutional approaches, 
organizational and administrative support, management practices and policies, and institutional 
uses and impacts related to student assessment. But even more important is understanding how 
these domains influence the likelihood that student assessment will make a difl'erence in 
institutional performance. That is, which external influences, institutional approaches to 
assessment, organizational and administrative support patterns, and assessment management 
practices and policies are most likely to promote the use and positive impacts of student assessment 
information? 

In this section, we examine three key relationships. First, we discuss how external 
influences — the state and regional accrediting agencies — influence an institution’s approach to 
.student assessment. Then we examine how external influences and institutional characteristics, 
approaches, suppon patterns, and management practices and polices affect an institution’s use of 
.student assessment data. Finally, we examine how these domains lead to positive institutional 
impacts from student assessment. The analyses in this section use the indices discussed in previous 
sections. For detailed information on the indices, please refer to Appendix B. 

External and Internal Influences on Student Assessment Approaches 

Are external forces more influential than internal forces in determining how doctoral and 
research institutions approach student assessment? We used three multiple regression models to 
compare the influences of state characteristics related to student assessment, regional accreditation 
membership, and institution-wide support patterns for student as.sessment on the extent of an 
institution’s u.se of three approaches to student assessment; cognitive assessment (the extent to 
which data are collected on measures of students’ cognitive performance); affective assessment (the 
extent to which data are collected on measures of .students’ affective development or pertormance); 
and post-college assessment (the extent to which data are collected on measures of students' post- 
enrollment performance). In these three models, the predictor variables stem from three domains: 
state assessment approach, accrediting region, and institution-wide support. The predictor 
variables are shown in Table 6 and defined in detail in Appendix B. These analysis were re.stricted 
to public comprehensive institutions. 



Predictors of Cognitive Assessment 

Which of the variables has the most influence on whether a doctoral and research institution 
will assess students’ cognitive abilities? The first two columns of Table 6 show how institutional, 
state and regional accrediting influences are related to the extent to which these institutions collect 
data on students’ cognitive performance. This model predicted a moderate proportion of the 
variance in institutions’ collection of cognitive data, accounting for 22% of the variance. 



Table 6. The Influence of Institution-Wide Support, State Assessment Approach, and 
Accrediting Region on the Extent of Institutional Approach to Student Assessment for Public 
Doctoral and Research Institutions 

Extent of Institutional Approach 



Cognitive Assessment Affective Assessment 
Beta AR' Beta AR' 



Post-College 

Assessment 

Beta AR' 



R- 



21 ** 



. 06 * 



Institution-Wide Support 
Mission emphasis 

Administrative & governance .24* .06 

activities 

Administrator & faculty 
support 

Conduct for internal .24* .06 .29*'"' .11 .25* .06 

improvement 
Conduct for state 
Conduct for accreditation 
State Assessment Approach 
Authority structure 

Form of state as.sessment .2 1 * .04 

initiative 

Common indicators/outcomes -.25** .06 

Accrediting Region 
Middle States 

North Central .24* .10 

New England 

Northwest*** 

Southern 

Western 

*/? < .05; **p < .01 

***Since “accrediting region" was a categorical variable. Northwestern accrediting region was left out of this 
regression because its effect on cognitive competencies, ba.sed on ANOVA analysis, was clo.sest to the mean. 



Only three strong predictors emerged, one each from the domains of institutional, state and 
accreditation variables considered in this model. Of these, regional accreditation membership is the 
strongest influence. Institutions in the North Central accrediting region are more likely to assess 
students’ cognitive abilities than institutions belonging to the other accrediting regions (accounts 
for 10% of the variance). This relationship may reflect variations in the length of time different 
accrediting regions have included student assessment criteria within their reporting requirements 
for institutions as well as differences in the specific emphases of those criteria. 



Doctoral and research universities that view internal institutional improvement as an 
important purpose for conducting student assessment collect the most extensive cognitive data 
(accounts for 6% of the variance). This finding lends support to the importance of engaging in 
student assessment for internal purposes as a means of promoting assessment efforts. 

Finally, doctoral and research universities with state mandates requiring the use of common 
.student asse.ssment indicators across institutions report less extensive collection of cognitive data 
(accounts for 6% of the variance). Interpretation of this relationship is speculative. One possibility 
is that state-level agents often do not require indicators related to cognitive competencies, hence 
institutions do not focus their efforts on collecting these data. 

Predictors of Affective Assessment 

The middle two columns of Table 6 show the relationship of variables in the institutional, 
state and regional accrediting domains to the extent to which doctoral and research institutions 
collect data on .students’ affective development. The variables in this model work reasonably well 
as predictors of affective assessment: the total model explains 21% of the variance in affective 
assessment efforts among doctoral and research institutions. Once again, only three variables are 
.stati.stically significant predictors in this model. 

The two strongest predictors come from the domain of institution-wide support. Doctoral 
and research in.stitutions that conduct assessment for internal improvement purposes are likely to 
collect more extensive affective data than those for whom this is a less important reason for 
engaging in a.ssessment (accounts for 1 1% of variance). The number of institution-wide 
administrative and governance activities used to promote student assessment is also positively 
related to collecting affective data (accounts for 6% of the variance). 

One state-related variable operates as a strong predictor. The strength of the state-level 
assessment initiative is positively related to collecting affective data (accounts for 6% of the 
variance). That is, doctoral and research universities in states whose assessment initiatives are 
guided by statute and policy, or by statute only collect more extensive affective data than those with 
•State-level assessment initiatives in the form of policy only or with no state plan for assessment. 
This suggests that for the.se institutions the authority of the state-level initiative does influence their 
as.se.ssment efforts. 

Predictors of Po.st-College Assessment 

The final two columns of Table 6 show the relationship of variables in the institutional, 
state and regional accrediting domains to the extent to which doctoral and research institutions 
collect data from their former students (post-college assessment). The variables in this model do 
not work well as predictors of post-college assessment: the model explains only 6% of the variance 
in these assessment efforts among doctoral and research universities. Only one variable operates as 
a significant predictor. Conducting assessment for internal improvement is predictive of the extent 
to which doctoral and research institutions collect post-college data. 

The poor ability of this model to predict institutions’ collection of post-college a.ssessment 
data is partly attributable to our earlier finding that few doctoral and research institutions have 
mounted extensive efforts to collect these data. This finding also suggests that institutions' 
decisions to collect post-college data from their students are influenced by variables not included in 
this model. 

Institution-wide support strategies, in particular, conducting asse.ssment for internal 
improvement, are strong predictors in all three student assessment approach models. This suggests 
that institutions themselves can influence the extent to which they collect various types of 



assessment data. Characteristics of state-level initiatives for student assessment are also related to 
the extensiveness of data collection efforts. Doctoral and research institutions are distinctive among 
institutional types in this respect. This observed relationship between state variables and 
institutional data collection is somewhat unexpected, both in terms of previous research suggesting 
that doctoral and research institutions are unresponsive to state-level efforts (Jemmott, 1992/1993: 
Ory & Parker, 1989; Scott, 1991) and findings reported earlier in this report (for example, that 
state requirements are a comparatively unimportant source of influence on their decisions to engage 
in student assessment). Finally, regional accreditation membership figures prominently as an 
influence in collecting cognitive but not affective or post-college data. 

Influences on Using Assessment Information in Academic Decisions 

How do doctoral and research institutions effectively promote and support the use of 
student assessment information in academic decision making? Within the domain of academic 
decision making, we created two indices reflecting the use of student assessment information; 
educational decision making and faculty decision making (these indices are described in detail in 
Appendix B and section seven). We used multiple regression to examine the influence of external 
forces, institutional size, institutional approach to student assessment, institution-wide support, 
and as.sessment management practices and policies on using student assessment information in ■ 
educational and faculty decisions. The predictor variables in this model are defined in detail in 
Appendix B. Table 7 presents the results of these regression models for doctoral and research 
institutions. 

Influences on the Use of Assessment Information in Educational Decisions 

The educational decisions model works very well for doctoral and research universities, 
explaining almost half (47%) of the variance in the influence of student assessment data on 
educational decisions. Significant predictor variables come from the four institutional domains: 
institutional characteristics, approach to as.se.ssment, institution-wide support, and asses.sment 
management practices and policies. 

Of these, the domain of assessment management practices and policies contributes the 
strongest influences. The extent to which doctoral and research institutions have policies providing 
professional development on student assessment for faculty and academic administrators has the 
strongest relationship to using assessment data in educational decisions (accounts for 19% of 
variance) followed by policies linking student as.sessment to faculty evaluation and rewards 
(accounts for 11% of variance), and policies encouraging student involvement in assessment 
activities (accounts for 3% of variance). As findings in section six showed, doctoral and research 
institutions report moderately extensive u.se of student involvement policies, but made 
comparatively little use of policies for professional development and rarely considered assessment- 
related criteria in faculty evaluation decisions. The use of the.se assessment management policies 
and of as.sessment information in educational decisions are both indicative of a strong internal 
commitment to student assessment. 

One variable from the assessment approach domain functions as a very strong predictor. 
The number of studies conducted of the relationship of student performance to institutional 
experiences is positively related to using as.sessment data to make educational decisions (accounts 
for 7% of variance). This is a logical relationship; conducting such studies should provide 
institutions with relevant data to consider in decisions concerning academic planning, instructional 
practices and academic support services — all decisions reflected within the educational decisions 
index. 



Table 7. The Influence of External Factors, Institutional Context. Institutional Approach. 
Institution-Wide Support, and Management Practices and Policies on Using Student 
Assessment Information in Educational and Faculty Decisions for Doctoral and Research 
Institutions 

Educational Decisions Beta Faculty Decisions Beta AR~ 



Adjusted R' .4 7'^'^ .04'^'^ 

External Influences 
Middle States accrediting region 
North Central accrediting region 
New England accrediting region 
Southern accrediting region 
Western accrediting region 
State initiative for student assessment 
State approach to student assessment 
Accrediting influence 
Institutional Context 

Control ( 1 = public, 2 = private) 1 5"^' .02 

Enrollment 

Institutional Approach to Student Assessment 
Cognitive assessment 
Affective assessment 
Post-college assessment 
Number of instruments 
Student-centered methods 
External methods 

Total assessment studies .22*'^ .07 



Institution-Wide Support for Student Assessment 
Mission emphasis -.16'’- .02 

Conduct for internal improvement .16* .02 

Conduct for state 
Conduct for accreditation 
Administrative & governance activities 

Administrator & faculty support .14* .02 

Formal centralized policy 
Institution-wide planning group 
Conducted evaluation of assessment process 
A.ssessment Management Practices and Policies 



Academic budget decisions 
Computer support 
Access to information 



Distribution of reports 

Student involvement .17* .03 .21* .04 

Professional development .28** .19 



Student affairs training 

Faculty evaluation' .26** .11 n/inc 

Academic planning and review • n/inc 

*/)< . 05 ; **/? < .01 

The factor "faculty evaluatioir was not entered into the regression model predicting u.se of .student as.se.ssnient information in faculty 
decisions as many of the items comprising these two factors were similar. 

‘The factor "academic planning and review" was not entered into the regression model predicting u.se of student as.sessment information in 
educational decisions as many of the items comprising these two factors were similar. 
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Three institution-wide support variables are predictive of using assessment information in 
educational decisions; each accounts for 2 % of the variance in educational decision uses. 
Conducting student assessment for internal improvement purposes and the perceived degree of 
administrator and faculty support for student assessment are positively related to using assessment 
information in educational decisions. Conversely, having an emphasis on undergraduate education 
and student assessment in the academic mission is negatively associated with this use. The 
relationships among the positive predictors in this domain and using assessment intomiatiori in 
educational decisions are straightforward. Doctoral and research institutions that engage in 
assessment with the intent of improving institutional performance, and those that have the support 
of faculty and administrators for assessment, are more likely to use assessment inlormation to 
shape academic decision making. Since this research focused on undergraduate student 
assessment, information collected from these assessment efforts may have little eltect on these 
primarily graduate education-oriented institutions' internal decisions. This may explain the negative 
relationship between mission emphasis and educational decision uses of assessment information. 

Finally, private doctoral and research institutions do not use assessment information in 
educational decisions to the same extent as their public counterparts (accounts for 2 % of the 
variance). No external influence variables are significant predictors of educational decisions, 
su 22 estins that this use of assessment information is primarily detenriined by internal influences. 

Influences on the U.se of Assessment Information in Faculty Decisions 

The use of assessment information in faculty decisions was not well explained by this 
model. Only one index emerged as a statistically significant predictor. The extent of policies 
encouraging student involvement in assessment activities accounted for 4 % ot the variance in this 
outcome measure. This result is not surprising, given that doctoral and research institutions rarely 
use student assessment information to make decisions concerning faculty promotion, tenure, salary 
or merit increases (refer to .section seven). 

Influences of Student Assessment Information on Internal and External 

Institutional Performance 

How does the use of student as.sessment information affect various internal and external 
institutional performance dimensions? In our survey, doctoral and research institutions reported 
whether, and the extent to which, student assessment has affected various aspects ot faculty and 
student performance and relationships with their external environment (these indices are described 
in Appendix B and section .seven). Using factor analysis, we created two indices of internal 
impacts from assessment: faculty impacts and student impacts. All of the items in the external 
impact section of the survey factored into one “external impact" index. We used multiple regression 
to examine the relationship of external forces, institutional approach to student assessment, 
institution-wide support patterns, and assessment management practices and policies to institutional 
impacts of student assessment information. We ran separate regression models tor each ot the three 
impacts. The predictor variables in these models are defined in detail in Appendix B. Table 8 
presents the re.sults of the three regression models for doctoral and research institutions. 

Predictors of Faculty Impacts 

What variables predict whether a doctoral or research university’s student assessment 
efforts will have an impact on its faculty members’ attitudes and activities? The first two columns 
of Table 8 show statistically significant predictors of faculty impacts. Five variables, all from 
domains of institutional variables, explained 34% of the variance in achieving faculty impacts from 
student assessment, the best fit for this model among all types of institutions. 



Table 8. The Influence of External Factors, Institutional Context, Institutional Approach, 
Institution-Wide Support, and Management Practices and Policies on Faculty, Student, and 
External Impacts for Doctoral and Research Institutions 

Faculty Impacts Student Impacts External Impacts 
Beta AR~ Beta AR~ Beta AR~ 

Adjusted ' .34^^ ^6^^ . 26 ^^ 

External Influences 
Middle States accrediting region 
North Centra] accrediting region 
New England accrediting region 
Southern accrediting region 
Western accrediting region 
State initiative for student assessment 
State approach to student assessment 
Accrediting influence 
Institutional Context 

Control (1 = public, 2 = private) -.15* .05 -.20** .03 -.23*^ .05 

Enrollment 

Institutional Approach to Student As.sessment 
Cognitive assessment 
Affective assessment 

Post-college assessment .17* .05 

Number of instruments 
Student-centered methods 
External methods 

Total assessment studies .35** .16 



Institution-Wide Support for Student Assessment 
Mission emphasis 
Conduct for internal improvement 
Conduct for state 
Conduct for accreditation 



Administrative & governance activities 
Administrator & faculty support 
Formal centralized policy 
Institution-wide planning group 
Conducted evaluation of a.ssessment process 
Asse.ssment Management Practices and Policies 


.26** 


.12 


.20** 


Budget decisions 
Computer support 
Access to information 






.18* 


Distribution of reports 
Student involvement 


.19* 


.06 


.16* 


Professional development 
Student affairs training 
Faculty evaluation 
Academic planning and review 


.25** 


.07 


.42** 



*/? < .05; **p < .01 



.03 



.03 

.05 



.22 .25** .05 
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Four variables/indices were strong positive predictors of faculty impacts. The strongest of 
these was from the institution-wide support domain — the number of institution-wide 
administrative and governance activities undertaken to promote student assessment (e.xplains 129r 
of the variance). Fo7 faculty, these activities may signal the degree to which institutional leaders 
support and value undergraduate student as.sessment. 

Two assessment management practices and policies are also important determinants of 
whether an institution’s assessment activities will affect its faculty. Doctoral and re.search 
institutions that offer professional development on student assessment to their faculty and academic 
administrators, and that distribute summary reports of assessment information to a wide array of 
internal and external personnel are more likely to report their assessment activities have impacted 
faculty (respectively account for 7% and 6% of variance). 

One assessment approach variable positively predicts whether faculty impacts will result 
from assessment information: the extent to which the institution collects assessment data from 
former students (accounts for 5 % of variance). These data may provide faculty with important 
information concerning the relevance of their curricula or the effectiveness of various teaching 
practices and learning experiences employed. 

Conversely, being a private rather than a public doctoral and research institution is the only 
significant negative predictor of positive faculty impacts from student assessment (accounts for 5% 
of the variance). This suggests that faculty within private institutions are less responsive to 
information stemming from student as.sessment efforts. 

Predictors of Student Impacts 

What variables predict whether a doctoral or research university’s student assessment 
efforts will have an impact on the performance of its students? The middle two columns of Table 8 
suggest that institutional choices and activities from the domains of institutional characteristics. 
a.s.se.ssment approach, and assessment management practices and policies intluence the likelihood 
of achieving student impacts from assessment. This model works quite well, explaining 36% of 
the variance in .student impacts of student assessment. 

The domain of as.sessment management practices and policies contributes the strongest and 
most predictors of student impacts from assessment. The strongest of these is using assessment- 
related criteria in faculty evaluation policies (explains 22% of the variance). This finding underlines 
the potential importance of this management policy as a lever to enhance assessment information 
uses and impacts within doctoral and research institutions. Other significant predictors are the 
breadth of distribution of as.sessment reports (explains 5% of the variance) and provision of 
internal access to assessment information (explains 3% of the variance). 

From the approach domain, the number of institution-wide administrative and governance 
activities used to promote assessment is positively associated with achieving student impacts from 
assessment information (accounts for 3% of the variance). As noted in the discussion of faculty 
impacts above, these activities may provide an important signal of the importance that institutional 
leaders accord to student as.sessment. 

And again, being a privately controlled institution is the only significant negative predictor 
of student impacts (accounts for 3% of the variance). Private doctoral and research institutions are 
not subject to the same degree of scrutiny from state officials as public institutions. Thus they may 
feel less pressure to monitor student impacts of their assessment efforts. 



Predictors of External Impacts 



Lastly, what variables predict whether a doctoral or research institution's student 
assessment efforts will have an impact on its relationships with the external environment? The final 
two columns of Table 8 show the statistically significant predictors of external impacts. Only three 
statistically significant predictors emerge in this model, but together, they account for 269^ of the 
variance in achieving external impacts from student assessment. 

The strongest predictor comes from the domain of institutional approach to assessment. 
Doctoral and research institutions that conduct studies of the relationship between students' 
performance to their experiences with the institution are likely to report that their assessment 
activities have impacted their external relationships (explains 16% of variance). This finding 
reinforces the view of assessment scholars that institutions must do more than simply collect 
assessment data if student assessment is to promote institutional improvement. Doctoral and 
research universities that conduct such studies will be more aware of changes in student 
performance and of the institution's role in facilitating student performance. Presumably, these 
institutions will be better able to report positive findings regarding student perfonnance to their 
external constituencies. 

One assessment management policy is significantly predictive of achieving external impacts 
from student assessment. The extent to which assessment-related criteria are used in faculty 
evaluation and reward policies accounts for 5% of the variance in external impacts. The existence 
of such policies may demonstrate an institution's commitment to undergraduate teaching and 
assessment, with subsequent positive effects on external indicators of performance such as state 
funding allocation, .student applications, and institutional reputation. 

And once again, being a privately controlled doctoral or research university is a negative 
predictor of external impacts from assessment information. Compiued to their public counteiparts. 
private doctoral and research institutions often enjoy higher institutional prestige, make less use of 
state funds, and are able to be more selective in their admissions processes. This may account for 
the negative association between private institutional control and positive external impacts from 
assessment. 



Summary Observations 

External forces do influence doctoral and research institutions’ approaches to student 
assessment. Membership within specific accrediting regions is associated with vru’ying levels of 
engagement in collecting cognitive data. And doctoral and research universities are unique among 
types of postsecondary institutions in terms of the relationships observed between the form and 
content of state-level assessment initiatives policies and the type and extent of assessment data 
these institutions collect. But institutional choices and activities are also influential. One internal 
strategy is particularly important in detemiining the extent of student data collected; conducting 
assessment for internal improvement purposes. 

If doctoral and research institutions are to benefit from engaging in student assessment, 
they should not only be collecting student data but also using these data to make academic 
decisions. Institutions that use student assessment data for academic planning, instructional 
development and other educational decisions tend to commit to their assessment 
process — evidenced by the strategies of studying the relationship between student performance and 
institutional experiences, engaging in student assessment for internal purposes, and achieving a 
positive degree of support for assessment from administrators and faculty. These institutions also 
make more extensive use of assessment management practices and policies to support their 
assessment efforts; most notably, policies that provide professional development for faculty and 
academic administrators, link assessment-related criteria with faculty evaluation decisions, and 
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encourage student involvement in assessment. Doctoral and research institutions rarely use 
a.ssessment information in faculty-related decisions; we can say little about the predictors of this 
assessment u.se. External factors do not play a significant role in either of these uses of assessment 
data. 



If engaging in student assessment is truly making a difference in doctoral and research 
institutions, they should report that it is having an impact on their internal perfomiance and external 
relations. A number of institutional choices and efforts are strong predictors of achieving impacts 
from assessment information. Overall, the domain of assessment management practices and 
policies wields the most influence on achieving impacts from assessment infomiation. Doctoral and 
research institutions may increase the likelihood of achieving positive impacts on their faculty, 
students, and relationships with external constituencies if they include assessment-related criteria in 
decisions concerning faculty evaluation and rewards, widely distribute summary reports of 
as.sessment information, offer professional development on assessment to faculty and academic 
administrators, and provide broad internal access to assessment information. These findings 
support the views of scholars such as Ewell (1984, 1987a, 1987b, 1987c, 1988a, 1988b, 1997) 
and Banta (Banta & Associates, 1993; Banta et al., 1996) who assert that management practices 
and policies are powerful institutional levers for promoting the impact of student assessment in 
colleges and universities. Doctoral and research institutions reporting positive impacts from 
a.s.sessment conduct studies of the relationship between student performance and various aspects of 
their institutional experiences, demonstrate leadership support for student assessment by enaeting 
institution-wide activities to promote assessment, and collect more extensive data on the post- 
enrollment performance of former students. 

One consistently negative predictor of achieving positive impacts from assessment 
information must be noted. Private doctoral and research institutions are less likely to monitor 
whether, and how, assessment information has affected the performance of their faculty and 
students, and their relationship with external constituents. In addition, no variables related to state- 
level assessment initiatives or regional accreditation membership appear to significantly intluence 
a.ssessment impacts. Together, these findings emphasize the need for administrators and faculty 
leaders within doctoral and research institutions to develop strategies and take actions that promote 
internal engagement in undergraduate student as.sessment. Efforts that are initialed from outside the 
in.stitution are likely to be less effective at encouraging these institutions to use information gained 
from assessment efforts to improve student and institutional performance. 



9. EXECUTIVE SUMMARY: RESEARCH RESULTS AND RECOMMENDATIONS 



Recommendations for doctoral and research institutions based on the results 
of our research are outlined in this section. This study and monograph 
highlight the importance of viewing student assessment from a critical 
institutional perspective and underscore three important realities: l)for .student 
assessment to enhance student, faculty, academic, and institutional 
performance, it has to be viewed as an institutional process, not ju.st a .series 
of student assessment activities: 2) a great deal of organizational, 

administrative, and academic activity is or can be invested in initiating, 
managing, and using student assessment; and 3) a systematic look at those 
institutional activities can enhance an institution 's ability to plan for and use 
student assessment effectively. 

Student Assessment: A Critical Institutional Perspective 

The intent of our study and this monograph is to highlight the importance of viewing 
student assessment from a critical institutional perspective and to underscore three important 
realities: 1) for student assessment to enhance student, faculty, academic, and institutional 
performance, it has to be viewed as more than just a .series of student assessment activities: 2) a 
great deal of organizational, administrative, and academic activity is or can be invested in initiating, 
managing, and using student assessment; and 3) a systematic look at those institutional activities 
can enhance an institution’s ability to use student assessment effectively. 

The data reported in this survey provide a national profile of what doctoral and research 
institutions are currently doing to support and promote the use of student assessment. This 
evidence provides insight into the activities institutions are doing extensively, those which merit 
greater attention, and those which make a difference in improving institutional performance. We 
begin with a summary of our research findings and recommendations concerning specific 
assessment activities conducted within research and doctoral universities. Then we discuss how 
doctoral and research institutions can use the Institutional Support for Student Assessment (ISSA) 
inventory and the Framework for Institutional Support for Student Assessment (Figure 1 of this 
monograph) as a guide for examining their student assessment process and functions. We conclude 
by suggesting that the results of the national survey used in conjunction with institutional self- 
examination can serve as the basis for redesigning or planning a student assessment process which 
can enhance an institution’s academic performance. 

Student Assessment in Doctoral, and Research Institutions: Results and 
Recommendations from the National Survey 

Institutions that have committed resources to assessing the development and performance 
of their undergraduate students assessment activities should be able to use the student assessment 
data they collect in organizational decision-making and to document impacts from their assessment 
activities. Doctoral and research institutions that are engaging in undergraduate student assessment, 
but do not feel that they are profiting from the process as much as they could be, may want to 
adopt the strategies of institutions that have reported using and being positively impacted by 
assessment information. Our research has demonstrated that most doctoral and research institutions 
have made only limited use of student assessment data and given little attention to monitoring 
impacts from assessment. However, they are subject to external demands for assessment and have 
adopted a wide variety of student assessment approach measures, institution-wide activities 
supporting student assessment, and assessment management practices and policies. Our research 
shows that external influences do play some role in encouraging institutions to conduct and use 
student assessment, but there is much that institutions themselves can do to initiate, support and 



benefit from their undergraduate student assessment efforts. Our research identified strategies from 
the domains of student assessment approach, institution-wide support tor student assessment, and 
assessment management practices and policies that are associated with doctoral and research 
institutions reaping greater institutional uses and impacts from undergraduate student as.sessment. 
Greater u.ses and impacts mean that not only the institution as a whole benefits, but that faculty and 
students benefit as well. In the following sections, we highlight the student as.sessment activities 
that doctoral and research institutions are currently engaging in to a great extent, the activities that 
they may want to augment, and the activities that are critical to enhancing the use of student 
assessment data in academic decisions leading to positive institutional impacts. 

Student Assessment Uses and Impacts 

If institutions are benefiting from their student assessment process, they should report that 
they are using the resulting assessment data in academic decisions and that the u.se of these data has 
a positive impact on both mternal processes and performance and the institution's relationship with 
external constituents. In this section, we describe the extent to which doctoral and research 
institutions are using collected student assessment data and the extent to which these data have had 
an impact on the institution. Suggestions for increasing institutional uses and impacts of student 
assessment are outlined in the following sections. 

U.ses of Student Assessment Information . Most doctoral and research universities are not 
using student a.s.se.ssment data to make decisions about educational processes or faculty issues. In 
fact, respondents from these institutions report the least influence ot assessment information on 
institutional decision-making of all types of institutions in our study. Our research did not attempt 
to discern why institutions are not making more extensive use of these data. Perhaps institutional 
decision-makers do not have sufficient data to inform such academic decisions. This may he 
particularly so if assessment data collection focuses on student inputs (for example, basic college- 
readiness skills or academic intentions of entering .students) or outputs (for example, graduation or 
employment rates of exiting students) without attempting to measure changes in student 
performance or development, or without examining the relationship between student perfomiance 
or development and facets of their experiences within the institution (for example, course-taking 
patterns, financial aid, advising policies). As.sessment data use may also be limited if institutions 
have not created a formal mechanism for incorporating this data in decision-making processes. 
Doctoral and research institutions that are collecting student assessment data hut are not using these 
data in institutional decision making should examine why this is the case. For example, is the type 
of data being collected not u.seful for informing decisions? Are there concerns about data quality? Is 
assessment data not easily accessible or not widely distributed to decision makers? Raising 
questions of this nature will assist institutions in understanding how they may increase the use of 
assessment information in academic decision making. 

Impacts of Student As.sessment Information . Similarly, doctoral and research institutions 
report very minimal institutional impacts from their student assessment data. We found that most 
institutions are simply not monitoring whether assessment information has had an impact. Of all 
institutional performance indicators considered in our survey, doctoral and research institutions are 
most likely to attribute favorable evaluations from their regional accreditation agency, changes in 
the teaching methods used by their faculty, and increased discussions of undergraduate education 
to .student assessment information. Some institutions have documented as.sessment-related 
increases in student retention and graduation rates, and student satisfaction. Doctoral and research 
institutions have least often monitored the impact of assessment on student grades and external 
examination performance or on other dimensions of their relationships with external constituents 
such as state or private .sector funding, student application and acceptance rates, and institutional 
reputation. Monitoring impacts is important if an institution is to evaluate the effectiveness of its 
student assessment efforts. As a first step, the institution must select the indicators of institutional 
and student performance it wants to monitor as well as the evidence it will use to measure these 



indicators. Which specific aspects of performance will be monitored, and the type of evidence 
used, will vary according to the institution's academic mission and its goals for student 
assessment. 

Regardless of the specific uses and impacts an institution may hope to achieve through 
student assessment, the remainder of this report suggests institutional responses to external 
influences, approaches to assessment, and organizational and administrative practices and policies 
that may promote institutional uses and impacts of student assessment data. 

External Influences 

External influences, such as state and accreditation requirements, can be strong motivators 
for institutions to engage in or increase their student assessment efforts. In addition, a variety of 
postsecondary organizations and external funding sources offer support for institutions' 
as.sessment programs. This section describes the nature of these influences on doctoral and 
research institudons and provides suggestions for deriving the most benefit from these external 
relationships. 

State Influences . Half of the doctoral and research institutions that responded to our survey 
are under the purview of a state mandate for student assessment. Three-quarters ot these 
institutions report that they worked together with state officials to develop the details ot the state- 
level a.s.se.ssment plan. It follows that one-quarter of institutions were not involved in this 
development. Our research suggests that where such opportunities exist, institutions should ti7 to 
participate in such planning at the state level. State-level assessment mandates that are Jointly 
developed by institutional and state officials have a greater potential to be fair, practical, and 
ultimately beneficial to institutions. Doctoral and research institutions with state assessment plans 
permitting them to develop and report their own student assessment indicators collect more 
extensive assessment data than institutions with state plans mandating the use ot common student 
assessment indicators across institutions. Further, our research shows that the importance 
institutions accord to meeting state reporting requirements as a purpose for engaging in student 
a.s.sessment is not significantly related to how extensively institutions collect assessment data, how 
much they use that data for decision-making, or how much they achieve positive impacts from 
using a.ssessment data. Said differently, institutions that conduct student assessment solely to 
satisfy state requirements are not likely to reap internal or external benefits from assessment. 
Doctoral and research institutions that must commit resources to student assessment because of 
state requirements are advised to also use these efforts tor internal improvement purposes. 

Accreditation Influences . The majority of doctoral and research institutions have completed 
a regional accreditation review requiring undergraduate student assessment. While these 
institutions are less likely than other types to report accreditation requirements influenced them to 
initiate student assessment efforts, most doctoral and research institutions credit these requirements 
with increasing their involvement in assessing students. Regional accreditation associations have 
some direct influence on institutions' student assessment activities. For example, doctoral and 
research institutions in the North Central accrediting region collect more cognitive assessment data 
than institutions belonging to other accrediting regions. However, institutional membership in a 
specific regional accreditation association is not significantly related to either using or achieving 
impacts from student assessment data. Institutions should be cognizant of the influence of their 
accrediting regions and must, of course, respond to their requirements for assessment. But 
doctoral and research institutions should also bear in mind that if responding to accreditation 
requirements is the only major purpose for engaging in student assessment activities, these 
activities will not be likely to have an impact on how extensively institutions collect assessment 
data, how much they use that data for decision-making, or how much they achieve positive impacts 
from using assessment data. Accreditation requirements notwithstanding, institutions that want to 
maximize assessment information uses and impacts are advised to make certain that the 



improvement of institutional performance (for example, improving academic programs and student 
achievement) is an important purpose underlying their assessment efforts. 

Other External Influences . In addition to state and regional accreditation requirements, there 
are other external resources doctoral and research institutions should consider when engaging in 
student assessment activities. Many doctoral and research institutions have made use of 
conferences and publications on student assessment provided by professional associations and. to 
a slightly lesser extent, regional accreditation associations. Although a variety of external agencies 
and programs offer grants to improve or support institutions' assessment practices, the majority of 
doctoral and research institutions have not received such grants. Doctoral and research institutions 
interested in augmenting their assessment efforts may want to explore these opportunities tor 
external support. 

Institutional Approach to Student Assessment. 

Institutional approach to student as.sessment refers to institutional decisions regarding the 
collection and analysis of student assessment infomiation. Dimensions along which assessment 
approaches can be differentiated include: the type and extent of student assessment data collected; 
the methods used to collect assessment data; and the analyses conducted and reported tor collected 
data. This section summarizes doctoral and research institutions' approaches to student assessment 
and offers recommendations for adopting assessment approaches that will lead to uses and impacts 
of student assessment information. 

Type and Extent . Most doctoral and research universities emphasize the collection of data 
on certain types of cognitive, affective and behavioral data from their currently-enrolled students 
such as academic progress, basic college-readiness skills, academic intentions, satisfaction, and 
experiences and involvement with the institution. These data are most often collected at one point in 
time during students’ enrollment and provide static measures of students' basic abilities, 
satisfaction and movement through the educational process that may be sufficient to satisfy 
accountability requirements. However, they provide institutions with less information concerning 
how their students are learning and changing while in college. Doctoral and research institutions 
collect less extensive information on their current students’ higher-order cognitive competencies, 
vocational and professional skills, general education and major field competencies, and personal 
growth and affective development, and collect comparatively less extensive data from their 
formerly-enrolled students, particularly regarding their civic and social roles. Collecting these types 
of data, at more than one point in time, would provide doctoral and research institutions with more 
substantive data concerning students’ cognitive, affective and behavioral development. In addition, 
our research shows that the more assessment data institutions collect from foimeiiy-enrolled 
students, the more likely their faculty are to be positively affected by assessment information. If 
they are to achieve maximum benefit from assessment efforts, doctoral and research institutions 
should review the type and extent of assessment data being collected and, where needed, broaden 
and deepen these collection efforts. 

Assessment Methods . Our survey results show doctoral and research institutions tend to 
use tests and other written instruments to collect assessment data. These tests and inventories are 
most often developed by institutions themselves, which may enhance their acceptance and use 
within the institution. Some units or departments make use of less traditional assessment methods 
such as capstone courses, portfolios of student work, observations of student performance, and 
surveys or interviews with withdrawing students. Institutions should encourage even greater use 
of these alternative methods of collecting student assessment data within their departments. These 
alternative, methods allow the assessment of higher-order skills such as application of facts, 
synthesis, and evaluation. They also tend to promote faculty involvement in assessment through 
their participation in designing one, administering their method, and interpreting the data collected 
through them. 



Assessment Studies . Although student learning and development are affected by students' 
internal motivation and abilities and by circumstances external to the institution, a variety of 
institutional factors can also influence how students perform, such as the availability of academic 
support resources, time spent with faculty outside of the classroom, kinds of teaching methods 
used, and interventions from advisors or counselors. Compared to other types of institutions, 
doctoral and research institutions have a strong profile of conducting such studies; more than three- 
quarters of doctoral and research institutions in our study do so. Most often these studies examine 
the relationship of non-academic policies and practices (for example, admissions standards, 
residence arrangement, financial aid and extra-curricular activities) to students' academic 
performance. fToctoral and research institutions are less likely to examine the connection between 
the specific dimensions of the teaching/leaming environment (for example, exposure to specitic 
teaching methods, interaction with faculty) and student pertormance. The number ol studies 
conducted by institutions is a strong predictor of using assessment information to make educational 
decisions and of achieving positive external impacts from assessment information. Clearly, 
institutions that are not already doing so should commit resources to such studies if they want their 
as.sessment activities to produce useful information and positive results. Those doctoral and 
re.search institutions that do already conduct some studies of the intluences on student pertonnance 
are advised to evaluate whether these analyses include aspects of the teaching/leaming 
environment. 

Institution-Wide As.sessment Support Strategy 

Doctoral and research institutions have used a variety ot organizational and administrative 
support strategies to support their student assessment efforts such as including student assessment 
within the academic mission; determining the main purposes for which the institution is engaging 
in a.sse.ssment; sponsoring institution-wide activities to promote involvement in and support for 
a.ssessment; adopting an institutional plan or policy for student assessment; establishing processes 
and structures for planning and coordinating assessment; and evaluating the assessment process. 
This .section summarizes the institution-wide support strategies adopted by doctoral and research 
institutions and offers recommendations regarding specific aspects of support strategy that enhance 
the likelihood of achieving institutional benefits from student assessment information. 

Mission Emphasis . One way to determine the degree of support for an activity within an 
in.stitution is to examine whether the activity is mentioned in its mission statement. While most 
doctoral and research institutions emphasize excellence in undergraduate education as part of their 
mission statements, less than one-third mention educational outcomes intended for their students 
and fewer still cite the importance of student assessment as an institutional activity. Our research 
.shows that doctoral and research institutions that have included an emphasis on undergraduate 
education and student assessment in their academic mission statements are slightly less likely to use 
asse.ssment information in educational decisions. Clearly, the inclusion of these emphases within 
the academic mission is not, in it.self, enough to promote internal support for assessment. 
Institutions that have incorporated statements regarding the importance of undergraduate education 
and as.sessment in their mission statement but are not extensively engaged in assessment or using 
collected assessment data are advised to review the possible reasons for this inconsistency. What 
other organizational and administrative practices or policies been developed to encourage internal 
engagement in assessment efforts? Have assessment processes been formally linked with 
institutional processes for decision making? The following sections review practices and policies 
that can promote greater support for and use of assessment. 

Assessment Purposes . Similarly, support for assessment can be examined by considering 
an institution’s purpose for engaging in student assessment. Doctoral and research institutions 
report that improving undergraduate education and preparing for institutional accreditation are both 
important purposes for engaging in assessment, although they accord less importance to meeting 
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external mandates from state officials and regional accreditors than do other types of institutions. 
Our research found that the purpose for which assessment is being conducted is significantly 
related to the institution’s approach to assessment and using assessment information. Doctoral and 
research institutions that report promoting internal improvement is an important purpose underlying 
their assessment efforts are more likely to collect extensive assessment date and to use assessment 
information for institutional decisions. Conversely, engaging in student assessment to meet state 
and regional accreditation requirements is not a significant predictor of a.ssessment approach, uses 
or impacts. Therefore, if doctoral and research institutions want to benefit from their student 
assessment activities, they should examine their purposes for engaging in student assessment and 
be clear that these go beyond meeting external demands. 

Institution-Wide Activities . Another measure of support for student assessment is the 
number of institution-wide administrative and governance activities used to promote student 
assessment. Many doctoral and research institution respondents report that they offer workshops 
on student assessment for academic and student affairs administrators and have a faculty 
governance committee that addresses assessment issues. Roughly one-third hold institution-wide 
initiatives on assessment, provide incentives or rewards to encourage academic units to engage in 
student assessment, and include student representatives on asses.sment committees. They rarely 
offer incentives or rewards to individual administrators who promote student assessment or 
involve tru.stees assessment committees. Doctoral and research institutions should review the 
nature and number of institution-wide activities used to promote internal involvement in and 
support for assessment, and where indicated, include additional avenues. Not only does offering 
such activities demonstrate leadership support for assessment, but our research found that offering 
a greater number of administrative and governance activities to promote assessment is a significant 
predictor of the extensiveness of these institutions' data collection efforts and of documenting 
positive impacts on faculty and student from assessment. 

Administrator and Faculty Support . Our survey also asked how supportive various internal 
constituents were of student assessment. While doctoral and research institutions report moderately 
high levels of support for all internal constituencies considered, senior administrators — the chief 
executive officer, academic affairs and student affairs administrators — are perceived as 
comparatively more supportive of assessment than boards of trustees, faculty and students. The 
perceived degree of administrator and faculty support is positively related to the intluence of 
assessment information on educational decisions within doctoral and research institutions. It is 
advisable for institutions to periodically measure the degree of internal support for assessment 
among internal constituent. groups as a useful means to gauge the effectivene.ss of efforts to 
promote assessment and to encourage more extensive use of assessment information in institutional 
decision making. 

Institutional Plans and Policies for Assessment . The types of plans and policies institutions 
develop for assessing students are also indicative of their support for student assessment. Most 
doctoral and research institutions have some type of institutional plan or policy for undergraduate 
student assessment in place. They are more likely to adopt a decentralized policy in which academic 
units or programs are expected to assess students but determine for themselves the specific form 
assessment activities will take than a centralized policy in which specific assessment activities are 
required of all units or programs. However, compared to all institutions in our study, doctoral and 
research universities are less likely to have a formal assessment plan or policy of any type. The 
absence of an assessment policy may signal to internal con.stituents that the institution does not 
consider student assessment a priority. Accordingly, doctoral and re.search institutions that do not 
have an institutional plan or policy for student assessment should consider some form of policy. 
Our findings suggest that a decentralized assessment policy is most consistent with the overall 
governance style of these often large and organizationally complex institutions. 
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Planning and Coordinating Assessment . Only half of doctoral and research institutions in 
our study have some type of institution-wide planning group for student assessment. When such 
planning groups exist, they are most often staffed by faculty, academic administrators, institutional 
researchers and student affairs administrators. Executive responsibility for planning student 
assessment is usually positioned within academic affairs, while oversight of day-to-day assessment 
activities is most often assigned to academic affairs or institutional research personnel. It .seems that 
planning and coordinating student assessment is mainly the responsibility of academic 
administrators within many doctoral and research institutions. Faculty, student affairs personnel 
and students are comparatively less involved. Conceivably, involving these constituencies in 
planning assessment activities would increase their understanding of the purposes of as.sessment 
and encourage their participation in assessment activities. Doctoral and research institutions are 
encouraged to review and reconsider their internal patterns of structuring and involving various 
groups in planning and coordinating assessment efforts. 

Evaluation of Assessment Process . Whether an institution has evaluated its assessment 
approach is also an indication of the importance it accords to .student assessment as an institutional 
activity. More than half of the doctoral and research institutions responding to our survey have not 
evaluated their assessment approach and one-third indicate having no plans to do .so. All 
institutions should consider evaluating their assessment plans, policies and processes in order to 
determine whether their efforts are meeting their intended objectives and whether the effort being 
expended to asse.ss students is benefiting the institution. 

Assessment Management Practices 

Assessment management practices refer to specific practices intentionally devised by 
institutions to manage their student assessment efforts. Four specific areas of practice were 
identified as potential influences on student assessment: academic resource allocation, student 
information sy.stems, internal access to student information, and distribution of assessment reports 
and .studies. This section summarizes the assessment management practices used by doctoral and 
research institutions and provides suggestions regarding practices that maximize institutional uses 
and impacts of student assessment information. 

Academic Budget . Doctoral and research institutions rarely allocate resources to academic 
units based on indicators of their student assessment activities or results. Assessment scholars 
contend that linking assessment information or efforts to budgetary decision making processes in 
the institution is a powerful means of establishing an effective student assessment program. 
Developing or augmenting a linkage between academic resource allocation and assessment-related 
performance indicators may provide a clear message regarding the importance placed on student 
assessment by leaders within doctoral and research institutions. 

Computer Support . Doctoral and research institutions report having limited information 
system capabilities for student assessment. Only half have a student information system that can 
track students from application through graduation. Even fewer have a system that includes student 
performance indicators or that is integrated with other in.stitutional databases. Along with 
improving the management of student assessment data, creating relational databases in which 
student assessment information can be linked to other institutional data would facilitate conducting 
analyses of the relationship between students’ performance and their experiences within the 
institution. As discussed above, conducting studies of this nature is a strong predictor of using and 
achieving positive impacts from assessment information. Doctoral and research institutions should 
examine the capabilities of their information system for student assessment. 

Access to Assessment Information . Most doctoral and research institutions do provide a 
variety of internal constituents with access to as.sessment information on individual students — 
particularly institutional research personnel, academic administrators and faculty advisors. 
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Providing such access makes it easier for internal personnel to inter\'ene with or give feedback to 
students regarding their performance. We found that providing internal access to assessment 
information on individual students is a significant predictor of achieving positive impacts from 
assessment on indicators of student performance. This information is less often accessible to 
.student affairs professionals. Doctoral and research institutions may wish to review this practice. 

Asse.ssment Report Di.stribution . Distributing reports that summarize assessment 
information and studies is also an important assessment management practice. Doctoral and 
research institutions that distribute assessment reports to a wide array of internal and external 
constituents are more likely to report positive faculty and student impacts from assessment 
information. Repon distribution helps to ensure that internal and external personnel are kept w'dl 
informed of the institution's assessment effons, student performance and institutional improvement 
.strategies. While most doctoral and research institutions regularly distribute assessment reports to 
traditional internal constituents such as academic administrators, faculty, and student affairs 
professionals, very few provide these repons to students, the general public or employers. 

Doctoral and research institutions are advised to examine their patterns of assessment report 
distribution. 

Assessment Management Policies 

A.ssessment management policies refer to institutional policies devised to both suppon 
student assessment and to direct the use of student assessment information. Five content 
dimensions of assessment management policies were examined; student involvement in 
assessment; professional development on assessment for faculty and academic administrators; 
training in assessment for student affairs personnel; faculty evaluation and rewards; and academic 
planning and review processes. This section summarizes the assessment management policies used 
by doctoral and research institutions and provides suggestions regarding policies that maximize 
institutional u.ses and impacts of student assessment information. 

Student Involvement . Doctoral and research institutions report that many of their 
departments require students to participate in assessment activities and provide students with 
information regarding the purposes of these activities, but few provide students with feedback on 
their as.sessment results or incentives for participation. Policies encouraging student involvement in 
assessment activities are an important mechanism for promoting internal engagement in 
as.sessment. Obviously, collecting useful student assessment information depends in large part 
upon the willingness of students to be involved in as.se.ssment activities. Providing explanations of 
the purpose of assessment activities and individual feedback to students regarding student 
performance should enhance students' motivation to participate. Further, our research shows that 
doctoral and research institutions that have developed more extensive policies on student 
involvement make significantly greater use of a.ssessment information in educational and faculty- 
related decisions. If student involvement policies do not exist or do so only within a few 
departments, doctoral and research institutions should consider extending the breadth of these 
policies. 

Professional Development . Doctoral and research institutions offer professional 
development on student assessment to their faculty and academic administrators. However, they 
offer less extensive policies of this nature than do other types of institutions. These policies are 
more likely to involve activities that do not significantly affect time in the cla.ssroom (for example, 
funds to attend assessment conferences or workshops on assessment) than those that involve 
greater expenditure of administrative and faculty time (for example, paid leaves or course reduction 
to develop or improve assessment practices). While understandable from a fiscal perspective, this 
emphasis on providing professional development support of shorter duration may not provide 
faculty with adequate time to leam about or develop new assessment techniques. Offering a variety 
of professional development opportunities should increase the level of faculty and administrative 
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involvement in and suppon for student assessment. Our research found doctoral and research 
institutions that offer more extensive professional development on assessment to their faculty and 
academic administrators are more likely to use assessment information in educational decisions (for 
example, designing or revising academic programs, curriculum or support ser\ ices) and to repon 
that assessment information has had a positive impact on faculty. Doctoral and research institutions 
should review and consider increasing the array of professional development opportunities on 
student assessment being offered to their academic personnel. 

Student Affairs Training . When planning professional development opportunities for 
student assessment, doctoral and research institutions should not neglect their student attairs stall 
and administrators. Given their frequent contact with students, these personnel can play an 
important role in planning assessment activities and interpreting how assessment data should shape 
institutional policies concerning students. Yet. on the whole, doctoral and research institutions 
provide only limited training on assessment to their student affairs personnel, doing so in only a 
few departments. This policy area deserves further examination and diagnosis. 

Faculty Evaluation and Rewards . Many doctoral and research institutions say they 
encourage their faculty to assess student learning, but it is rare for departments to have policies that 
include assessment-related criteria in faculty evaluation and reward decisions. Although the 
wisdom of this linkage has been debated in the assessment literature, doctoral and research 
institutions that do include assessment-related criteria in their faculty evaluation and reward 
proce.sses are also more likely to use assessment information in making educational decisions and 
also more often report positive impacts from assessment on their students and relationships with 
external constituents. Institutions may understandably be reluctant to tie faculty evaluation and 
rewards to assessment results or indicators of student performance, but there are many other 
options to consider. For example, institutions can include criteria concerning faculty scholarship in 
assessment, evidence of using as.sessment to improve teaching and learning, or participating in 
student assessment groups or committees. Given the traditional primacy of research and 
scholarship activities in determining faculty rewards in doctoral and research institutions, 
developing policies that link faculty participation in student assessment activities to promotion and 
salary decisions may offer an especially powerful means by which these institutions can encourage 
faculty to use assessment to improve teaching and learning. 

Academic Planning and Review . Doctoral and research institutions make fairly extensive 
use of student assessment infomiation in their academic planning and review processes, 
particularly those processes involving the review of academic departments, programs, or 
curriculum. Institutions that incorporate assessment data in academic planning and review 
proces.ses do not leave the use of as.sessment information to chance and thus may be more likely to 
reap positive impacts from their assessment information. Doctoral and research institutions are 
encouraged to build in formal linkages between the collection and analysis of student assessment 
data with specific academic planning, review and decision making processes. 

Inventorying the Institutional Student Assessment Process 

Regardless of an institution’s history of and support for student assessment, it is important 
for an institution to take stock of what it is currently doing. The Institutional Framework, presented 
in section two (Figure 1 of this monograph) provides an institutional perspective for such an 
examination. The ISSA inventory in Appendix A provides a useful quasi-objective instrument for 
identifying the specific dimensions and activities as.sociated with the student assessment process. 

Institutions are encouraged to identify a team of faculty and administrators most 
knowledgeable about and involved with student assessment to examine their institution’s activities 
on all the dimensions in the ISSA inventory. While most items are objective (for example, which 
types of measures are being used and how extensively), the actual pattern on campus may not be 



widely known. Other items are more subjective (for example, the institution's purpose tor 
engaging in student assessment) and can provide the basis for good discussion. 

This inventory of the institutional student assessment process then can be compared with 
national data for doctoral and research institutions which were presented in Appendix A and 
summarized in the previous sections of this monograph. This inventor>'ing and comparison 
process may highlight activities, practices, and policies not currently used; identify areas needing 
greater attention; or focus on inconsistencies in patterns of activity (for example, inconsistencies 
between stated purposes for student assessment and actual uses of student assessment data in 
academic decisions). 

For institutions with an extensive history of involvement with student assessment, such an 
inventory may serve as a useful basis for identifying new activities to be undertaken, for improving 
the existing activities and processes, for identifying issues or controversies that need to be 
addressed, "or for redesigning processes that may not be worth the current expenditure of effort and 
resources. For institutions with less experience with student assessment, the inventory may 
identify existing activities on which to build or help focus attention in the institution on the 
importance of student assessment in improving institutional performance. In either case, the 
inventory and self-evaluation process should help both to focus faculty and administrative attention 
on the importance of viewing student assessment as an institutional process and to deal with it 
more systematically — linking the various domains of assessment activity with institutional 
improvement. 



Planning for Student Assessment 

Student assessment in most higher education institutions has emerged, often sporadically, 
over the past decade due to the need to respond to an accreditation self study, a new state mandate, 
an academic administrator who promoted it, a faculty group who embraced it for their unit, or an 
institutional researcher or program review officer who was engaged in studies of student 
performance. While according to the data in this report, some institutions are beginning to develop 
a plan or policy for student assessment or have created a group for this purpo.se. there is little 
evidence of systematic planning that links student as.sessment to the external context and to internal 
institutional governance and management patterns; that develops organizational and administrative 
activities, practices, and policies to support it; and then uses collected assessment data for academic 
decisions and monitors their impact. Clearly there aie significant institutional differences and 
complex issues to be addressed if student assessment is to have positive effects. These deserve 
.some systematic, planned attention. The results of an institutional inventory can provide the basis 
for such an effort. 

While we do not advocate a cookbook or standardized approach to planning for student 
asses.sment (each institution needs to design its own planning approach to rellect its own 
governance and leadership styles and traditions), the Institutional Framework (Figure 1 in section 
two) and the institutional inventory provide a useful basis for redesigning or planning a student 
assessment process. Using that self-assessment, the following are planning issues that need to be 
addressed: 

External Influences 

1 . What is the nature of our state assessment process? How is it formulated and what are its 
requirements/implications for us? 

2. What are the accreditation requirements for our institutional and key professional accreditation 
bodies? 



3. What do some of our primary external constituents expect of our graduates? 

4. What are external sources of support (educational, financial, technical) for our student 
assessment efforts? 

Institution-Wide Support Patterns 

1 . What emphasis is placed on student assessment in our mission statement? What are the 
intended purposes of our institution's assessment efforts? 

2. What institution-wide administrative, governance, and academic activities support student 
as.sessment? Who are key leadership support groups and how are they involved? 

3. What is the nature of our institution-wide plan or policy for student assessment ? The role and 
membership of a coordinating or planning body? And the pattern of authority and 
responsibility for administering the process? 

4. Is the student assessment process to be evaluated? By whom? Using what criteria? 

Approaches to Student Assessment 

1 . Should we have an institution-wide or decentralized (by academic unit) approach to student 
assessment? 

2. What types of measures are to be used? How widely ? At what points in time? 

3. What types. of instalments and methods are appropriate ? What technical or professional 
suppoa does that require? 

4. What studies of the influence of students' educationally-related experiences on their 
performance are to be conducted? What repoas of student performance are to be prepared and 
distributed? 

A.ssessment Management Practices and Policies 

1 . What assessment management practices exist to guide student assessment ? (for example, 
resource allocation practices, infomiation systems, data access practices, and report 
distribution practices) 

2. What institutional policies promote the use of student assessment? (for example, policies on 
student involvement, professional development, student affairs training, faculty evaluation, 
and academic planning) 

Uses and Impacts of Student Assessment 

1 . How will we assure the use of student assessment information in educationally- and faculty- 
related academic decisions? 

2. How will we monitor the impact of student assessment on our students, faculty, academic and 
instructional patterns, and on our external relationships? 

While planning for student assessment may not resolve all of these questions, they should 

be addressed as should issues of the balance between the effort and resources required to maintain 

the institution's student assessment process and the educational and institutional benefits derived 



from assessment. When planning is combined with inventorying, these two processes can become 
a powerful mechanism for understanding and improving institutional student assessment 
endeavors. Inventorying existing student assessment processes is a first step toward understanding 
the nature and extent of an institution's student assessment activities. This can be followed by a 
planning process that considers the recommendations for doctoral and research universities that 
were presented in this .section. Using a systematic planning approach should increase the likelihood 
that student assessment. will contribute to improved institutional performance. 
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Inventory of 
Institutional Support for 
Student Assessment for 
Doctoral and Research Institutions 



For The Research Program on 
Institutional Support for Student Assessment 




N C P I 

' National 
Center for 
Postsecondary 
Improvement 



NCPI - Project 5.2 
University of Michigan 
Ann Arbor, Michigan 48109-1259 




An Introduction to the ISSA 



The Instinaional Support for Student Assesstnent Inventory (ISSA) was developed as pari of a national researcli 
program examining the Organizational and Adtninistrative Support for Student Assesstnent for the National Center 
for Postsecondary Improvement (NCPI). The ISSA is designed as an inslilulional inventory ot the organizational and 
administrative practices that have been designed and implemented to support the use o( student assessment on your 
campus. 

Institutional Support Practices are those organized activities, policies, and procedures that your insiilution has 
intentionally designed to enhance the practice of sludenl assessment. Student Assessment refers to those activities 
focused on measuring dimensions of student petfonnance other than traditional end of course grading. 

This national survey is designed to identify inslilulional support practices lor undergraduate sludenl assessmetn. The 
project also examines the factors influencing the adoption of various support practices and how those practices 
enhance the impact of sludenl assessment for institutional improvement. 

We understand that being selected for this survey will require a commitment of lime to complete and we appreciate 
your involvement. This instrument is also intended as an inslilulional self-assessment inventory to facilitate 
examination of your institution’s own organizational and administrative practices which support student assessment. 
We encourage each institution to use the survey in this manner. You will receive a summary report of survey 
respon.ses to all compare with your own inslilulional prollle. 

Completing the ISSA 

The main purpose is to obtain a profile of your institution's current approach to undergraduate student assc-^smeni 
and its support practices. The inventory may be completed by one individual or group of individuals who are most 
familiar with the patterns of undergraduate student assessmenl on your campus. It should take less than one hour to 
complete. 

• Please keep in mind that the questions refer to undergraduate education at your institution. 

• Respond to each item in the questionnaire to the best of your know'ledge. 

The questionnaire is coded to allow follow up only. Individual institutions will not he identilied in any analyses or 
reports. 

Return the completed questionnaire in the enclosed return envelope. Any questions concerning the survey can he 
addressed to the following: 



National Center for Postsecondary Improvement Project 5.2 
School of Education 
University of Michigan 
610 E. University. Room 2.239 
Ann Arbor. MI 4<S 109- 1259 
Phone: 734-647-2464 
Fax: 734-936-2741 
Email: ncpi.proj52@umich.edu 

Marvin W. Peterson, Project Director 



NCPI is funded by the U.S. Department of Education’s Office of Educational Re.search and Improvement under OERI 
grant number R309A60001 

© 1 997, The Regents of the University of Michigan 
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!. Institutional Approach to Student Assessment 



A . Type, Extent and Timing of Student Assessment 

We are inieresied in your institution's routine practices of collecting dilTcrent types of undergraduate student 
performance data, the extent to which they are collected, and when they are collected. For each.of the following 
content types of undergraduate student performance data: 

1) indicate the extent to which each type is collected 

2) for each type of data collected, check whether it is collected at entry, during enrollment, at exit, or 
a combination of these data collection points. 





Type 


Extent 






T i ni i n c 






Collecied 


Collected 


Collected 


Collected 


Collected Collected 






Not lor some 


for many 


for all 


at 


while at 




Currently Enrolled Students 


Collected studeni.s 


students 


students 


entry 


enrolled exit 


• 


(circle one number for each item) 


(Check iiii that apply tor each item) 






1 2 


3 


4 










D/R 


All 










1. Student academic intentions or 
















expectations 


3.11 


3.25 










2. Basic college-readiness skills 














• 


(reading, writing, mathematics, etc.) 
3. Higher-order skills (critical 


3.16 


3.44 










thinking, problem solving) 


1.98 


2,10 










4. General education competencies 


2.42 


2,55 










f 


5. Competence in major field 
of study (discipline- or 
program -.specific knowledge) 


2.53 


2.60 










6. Vocational or professional skills 


1.96 


2,1 1 












7. Personal growth and 
















affective development 
(values, attitudes, .social 
development, etc.) 


2.43* 


2.12 








# 


8. Student experiences and 
















involvement with institution 
9. Student satisfaction with 


2.69 


2.57 










institution 


2.84 


2.96 










10. Student academic progress 
(retention, graduation rates) 


3.91 


3.76 










Former Students 












§ 


1 1. Vocational or professional 
outcomes (career goals, job 
attainment or performance) 

12. Further education 


2.51 


2.72 










(transfer, degree attainment, 
graduate study) 

13. Civic or social roles 


2.48 


2.69 










(political, social or 
community involvement) 


1.94 


1.80 










14. Satisfaction and experiences 
with institution after leaving 


2.54 


2.63 
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B. 



Student Assessment Instruments 



Does your institution employ institutionally or externally developed instruments or tests for the following types of 
undergraduate student assessment information? (circle aU that apply for each item): 

Source of Instrument 





Noi 


Insiiiuiionally 


.Slate 


Commercial l\ 




used 


developed 


provided 


available 


Content of Instrument 


1 






' 


1. Student plans, goals, or expectations 


D/R: 249c 


50% 


3%r 


45 




All: 2l9c 


51% 


49f 


.32% 


2. Basic college-readiness skills 


D/R: 18%* 


52 %* 


13% 


43 %r* 


(reading, writing, mathematics, etc.) 


All: 69c 


38% 


1 !%■ 


67% 


3. Higher-order skills 


D/R: 44% 


36% 


3% 


32 %r 


(critical thinking, problem solving) 


All: 42% 


297r 


2% 


32% 


4. General education competencies 


D/R: 40% 


40% 


6% 


25 %r 




All: 32% 


40%' 


5% 


33% 


5. Competence in major field of study 


D/R: 21% 


69% 


7% 


39 %r 


(discipline- or program-specific knowledge) 


All: 19% 


64% 


1 2% 


.39% 


6. Vocational or professional skills (excluding 


D/R: 41% 


44% 


10% 


29 %r 


licensure exams) 


All: .^4% 


437r 


14%' 


25% 


7. Personal growth and affective development 


D/R: 23%=^ 


57 %* 


3%r 


35 %> 


(values, attitudes, social development, etc.) 


All: .38% 


39%' 


2%' 


.30% 


8. Student effort, experiences or involvement 


D/R: 14% 


69% 


6% 


29%, 


with institution 


All: 21%- 


60% 


49r 


24%' 


9. Student satisfaction with institution 


D/R: 3% 


76% 


10% 


36%: 




All: 4% 


73% 


8%' 


.35%- 


10. Alumni satisfaction and experiences 


D/R: 5% 


84% 


8% 


21 %> 




All: 10% 


78%' 


8%- 


15%- 



c. 



Other Student Assessment Methods 



To what extent does your institution use the following methods to collect undergraduate student assessment 
information ? (circle one number for each item): 



Other Student Assessment Methods 


Not 

used 

! 


Used in 
sonic units* 


Used in Used in 

most units all units 

4 






D/R 


All 


1 . Observations of student performance 
(simulations, demonstrations, lab) 




2.12 


2.26 


2. Student portfolios or comprehensive projects 




2.13 


2.10 


3. Student performance in capstone courses 




2.27 


2.15 


4. Student interviews or focus groups 




1.99 


1.84 


5. Transcript analysis 




1.92 


2.16 


6. External examination of students 
(licensure exams, external reviewers) 




2.02 


2.02 


7. Special surveys of or interviews with 








withdrawing students 




2.18 


2.40 


8. Alumni interviews or focus groups 




1.96 


1.90 


9. Employer interviews or focus groups 




1.84 


1.87 

1 


D. Student Sub-Populations 








Does your institution use different assessment methods 
(check one for each item): 


for the following 


sub-populations 


of undergraduate students? 




Different 




Same as Other Students 




D/R 


All 




1. Adult students 


6% 


\07c 




2. Part-time students 


5% 


57r 




3. Minority students 

4. ‘ Distance education students 


4% 


2%' 




21% 


22% 





’ “Unit” refers to academic areas such as departments, divisions, schools, or colleges. 
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E. Student Assessment Studies 



Does your insiilulion conduct studies of the relationship between the follow in^ experiences and students 
performance (check Ml Ripply)* 





D/R 


All 


I. Student course-taking patterns 


38 %* 


269; 


2. Exposure to different instructional or teaching methods 


25% 


21^A' 


3- Patterns of student-faculty interaction 


25%^ 


149; 


4. Extra-curricular activities 


37%^ 


249; 


5. Residence arrangements 


48%^ 


219; 


6. Student financial aid and/or concurrent employment 


44%^ 


309; 


7. Admission standards or policies 


60%^ 


429; 


8. Academic advising patterns 


26% 


269; 


9. Classroom, library and/or computing resources 


12% 


179; 


10. Do not study the relationship betw'. the above experiences & student performance 


22%^^ 


38^;; 


F. Student Performance Profiles or Reports 






Does your institution provide profiles or reports of appropriate student performance ini 


formation at 


the following 


levels of aggregation (check all apply ): 








D/R 


All 


I. Institution wide 


73% 


699; 


2. Schools or colleges 


73%^ 


:^v7c 


3. Academic programs or departments 


68% 


659; 


4- Special populations or subgroups/students 


59%^ 


469; 


5. By course or groups of courses 


32% 


369; 


6. Do not provide any reports 


9% 


1 19; 


II. Institutional Support for Student Assessment 






A. Institutional Emphasis 






1 - Your institutional mission statement explicitly (check ail apply): 








D/R 


All 


a. emphasizes excellence in undergraduate education as an institutional priority 86% 


829r 


b. identifies the educational outcomes intended for your students 


37%'*^ 


52% 


c. refers to student assessment as an important institutional activity 


14% 


19% 


d. does not explicitly mention any of the above 


11% 


.1 1% 



2. For how many years has your institution engaged in student assessment? 
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B. 



Purpose of Student Assessment 



The following are often intended purposes of an institution's undergraduate student assessment process. Please rate 
the importance of each for your institution, (circle cnie number for each item): 



Purpose 



1. Preparing institutional self-study 
for accreditation 

2. Meeting state reporting 
requirements 

3. Guiding internal re.source 
allocation decisions 

4. Guiding undergraduate academic 
program improvement 

5. Improving the achievement of 
undergraduate students 

6. Improving faculty instructional 
performance 

7. Other (brieflv describe): 



.\'o- 

Importance 

I 



Minor 

Importance 



D/R 

3 . 39 ^ 

2.50* 

2.45 

3.28 

3.27 

2.71* 



NKnleraie 

Importance 



All 

3.86 

2.89 

2.71 

3.43 

3.48 

3.02 



Ver> 

Important 

4 



c. 



Administrative and Governance Activities 



Institutions have introduced a variety of administrative or governance activities that address or promote student 
as.sessment. Does your institution engage in any of the following activities? (check a]i that apply): 







D/R 


All 


1. 


Annual presidential or other institution-wide initiatives, forums or 








.seminars on assessment 


3 1 % * 


41% 


2. 


Rewards or incentives for academic and student affairs administrators who 
promote use of student assessment in their unit 


18% 


6Vr 


3. 


Incentives for academic units to use student assessment information in 
their evaluation and improvement efforts 


34% 


217c 


4. 


Student a.sse.ssment workshops for academic and student affairs administrators 


59% 


56% 


5. 


Board of trustees committee that addresses student asse.ssment 


10% 


1 3% 


6. 


Faculty governance committee that addresses student assessment issues 


46%* 


58% 


7. 


Student repre.sentation on student assessment committees 


32% 


33% 
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D . Support for Student Assessment 

Use the scale below lo rate the degree lo which various groups within your institution support undergraduate student 
assessment activities (circle one number for each item): 





Very 


Somewhai • 


Neutral. 




Ver\ 




Unsupportive 


Unsupportive 


Unknown 


Supportive 


Suppc'rtive 




1 


2 




4 


5 






D/R 


ALL 






1 . Board of trustees 




3.68 


3.84 






2. Chief executive officer 




4 2 2 


4.41 






Academic affairs administrators 




4.43 


4,64 






4. Student affairs administrators 




4.29 


4.33 






5. Faculty governance 




3.51 


3.80 






6. Students 




3.24 


3.33 







E. Planning and Coordinating Student Assessment 

1 . Which of the following best describes your institution 's plan or policy for undergraduate student 



assessment? 

a. 


Your institution (check ajj that apply): 

has a formally adopted institutional plan or policy requiring 
specified undergraduate student assessment activities of ah academic 


D/R 


All 


b. 


units or programs 

has a formally adopted plan or policy for undergraduate student 
assessment in some academic units or pro iz ram areas (e.iz. izeneral 


32 %* 


50% 


c. 


education or academic majors) 

has a formally adopted institutional plan or policy requiring all 
academic units or programs to develop their own undergraduate 


18% 


19% 


d. 


student assessment plan 

has a formally adopted institutional plan or policy stipulating 
institution- wide activities to be conducted by a central committee. 


49%^' 


39% 


e. 


office, or officer 

has no formal plan or policy but academic units or programs are 
encouraged lo conduct their own undergraduate student assessment 


35% 


38% 


f. 


activities 

is currently developing a plan or policy for undergraduate student 


17% 


13% 




assessment 


13% 


17% 


o 

t * 


does not have an undergraduate student assessment plan or policy 
(SKIPTO E-6) 


11% 


4% 



2. Is there an institution-wide group (committee, task Ibrce, etc.) that is primarily responsible for ongoing 

planning and policy setting for undergraduate student assessment? (check one): 



a. 

b. 



yes 

no (SKiPTO Question E-5) 



D/R 
53 %* 



All 

70% 



3. If yes. who serves on this group? (check aU that apply): 







D/R 


All 


a. 


Chiefexecuiive officer 


39c 


1 3% 


b. 


Academic affairs adniinislralor(s)/siaff 


HS9r 


869^ 


c. 


Sludem affairs adminislralor(s)/siaff 


Si) 9c 


349; 


d. 


Inslilulional research adminisiraior(s)/siaff 


6\9c 


619; 


e. 


Academic review and evaluation adminisiraior(s)/siaff 


329c 


249; 


f. 


Student assessment administrator(s)/staff 


43 9c ^ 


329; 


a 

c- 


Faculty 


H9 9c 


919f 


h. 


Students 


369c 


339; 


i . 


Other 


10% 


129; 



Who has executive responsihiliry for or who chairs the institution -wide 


group responsible for 


the ongoi 


planning or policy-setting process for undergraduate student assessment? 


' (check ah that apply) 






D/R 


All 


a. Academic affairs administrator 


54% 


559c 


b. Student affairs administrator 


4% ■ 


19c 


e. Institutional research officer 


14% 


1 89;- 


d. Academic review and evaluation officer 


14% 


39; 


e. Student assessment officer (if separate) 


4% 


8% 


f. Faculty member 


28% 


319; 


ii. Other 


14% 


1 1% 



5. Who approves any changes in your inslilulion's plan or policies for undcrgraduaic sludcni asscssnicnl? 

(check a]_l that apply): 







D/R 


All 


a. 


Board of trustees 


8% 


17 ^;; 


b. 


Chief executive officer 


26%^ 


43 ^;;- 


c. 


Chief academic alTairs officer 


82% 


73 % 


d. 


Chief student affairs officer 


15% 


20 ^;; 


e. 


Institutional research officer 


15% 


18 % 


f. 


Academic review and evaluation officer 


17% 


8 % 


IT 
t ■ 


• Student assessment officer 


' 8% 


10 % 


J 


Student government 


0.8% 


\9o 


h. 


Academic senate or other faculty commitiee(s) 


26%* 


39% 


i . 


Faculty union 


0.8%^ 


4% 


k. 


Other 


5% 


14% 
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6 . 



Who has operational responsibility for your institution's day-to-day undergraduate student assessment 
activities (e.g.. instrument development, data collection, analysis, and reporting)? (check ajj that apply): 







D/R 


All 


a. 


Academic affairs administrator 


449c 


45% 


b. 


Student affairs administrator 


I99r 


209; 


c. 


Institutional research officer 


43 


45% 


d. 


Academic review and evaluation officer 


\6% 


9% 


e. 


Student assessment officer 


\49c 


15% 


i\ 


Faculty member(s) 


25% 


33% 


a_ 


Other 


\5% 


13% 


h. 


No one (SKIP TO QUESTION E8) 


5% 


3% 


To whom does the individual with operational responsibility for day- 


-to-day student assessment activities 


directly report? (check one): 










D/R 


All 


a. 


Chief executive officer 




29% 


b. 


Chief academic officer 


6H%^ 


56% 


c. 


Chief student affairs officer 


9% 


7% 


d. 


Institutional research officer 


2% 


3% 


e. 


Academic review and evaluation officer 


5% 


2% 


r. 


Other 


15% 


10% 


Is there an olTice which provides faculty consultation in using student assessment for instructional 




improvement or curriculum development? (check one): 










D/R 


All 


a. 


yes 


58%^ 


47%' 


b. 


no 


42%^ 


53% 


If ves. 


what is the name of the office? 






Evaluating Your Institution’s Student Assessment Plan 


or Process 




Has your institution evaluated its undergraduate student assessment process? (check one): 








D/R 


All 


a. 


yes. with a formal evaluation 


17% 


22%r 


b. 


yes, with an informal evaluation 


26% 


27% 


c. 


currently developing evaluation plans (SKIP TO SECTION III) 


24% 


29% 


d. 


not currently evaluating or planning to evaluate assessment process 33 % * 


2\% 




(SKIP TO SECTION) 







O 
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In evaluating your institution s student assessment process, which ot tlie lolUnving elements ot that 
process were reviewed? (check ^ that apply): 







D/R 


All 


a. 


your student assessment plan and policies 


73% 


81% 


b. 


the structure and responsibility for student asse.ssment 


56% 


64 % 


c. 


achievement of your institution's intended objectives for stud, assessment 


56 % 


709 ; 


d. 


reliability and validity of student assessment instruments and methods 


42%^ < 


549 ; 


e. 


quality of data analysis 


44% 


51 % 


f. 


use of student assessment information in institutional decision-making 


59% 


669 ' 




the problems encountered while conducting student assessment activities 


66% 


699 ; 


h. 


comparison of the costs and benefits of student assessment 


5%^ 


t j 



III. External Influences on Institutional Student Assessment Activities 
A . State Role (FOR STATE-FUNDED INSTITUTIONS ONLY: ALL OTHERS SKIP TO QUESTION III. B- 
1 . Was your state’s plan/requirement for student assessment primarily developed (check one)): 

D/R 

a. by state-level officials 13% 

b. through joint consultation between state officials and institutional representatives 36% 

c. no statewide plan or requirement for student assessment exists (SKIP TO III. B- 1 ) 5 1 % 



All 

16 % 

W7r 

469 ; 



State requirements for student assessment (check all that’apply): 



a. were an important reason for your institution to initiate undergraduate 
student assessment 

b. have increased your institution's involvement in undergraduate student 
assessment 

c. have not been a factor in your institution's undergraduate student 
assessment activities 

d- have been a negative influence on your institution's undergraduate 
student assessment activities 



D/R 

43% 

61% 

22 % 

6% 



All 

45 % 

62 % 

22 % 

4 % 



Your state's reporting requirements include (check ^ that apply): 



a. evidence that a student assessment plan is in place 

b. measurement of state-mandated student performance indicators 

c. institutionally-devised student performance indicators 

d. evidence of institutional use of student assessment information 



D/R 
61% 
60% 
61 %* 

39 %* 



All 

68 % 

64 % 

49 % 

52% 
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4. How has your slate higher education agency reviewed or evaluated your institution's undergraduate student 

assessment plan or process after it was implemented? (check all that apply): 







D/R 


All 


a. 


reviewed by slate officials 


38% 


429v 


b. 


review'ed using external reviewers 


14% 


169; 


c. 


required an institutional self-review' 


17% 


24% 


d. 


no post hoc review has occurred (SKIP TO QUESTION B- 1 ) 


56% 


« 449; 


5. The Slate review^ of your institution's undergraduate student assessment plan 


or process 


included (check ; 


that apply): 












D/R 


All 


a. 


review of your institution's student assessment process itself 


66% 


679; 


b. 


comparison of your institution's student performance record w'iih 








your past performance 


51% 


449; 


c. 


comparison of your institution's student performance record w'iih 








peer institutions 


34% 


369; 


d. 


comparison of your institution’s student perfomiance record w'ilh 








institutions in your stale 


54% 


^ 38^;; 


e. 


other thriellv describe) 


6% 


10^;; 


B. Regional 


Accrediting Role in Student Assessment 






I . Has your institution gone through a regional self study accreditation review which required 


undergraduate 


student assessment? (check one): 










D/R 


All 


a. 


yes 


78% 


S()% 


b. 


no 


22% 


297r 



2. Regional accreditation agency requirements for undergraduate student assessment (check aM that apply): 



3. 



a. were an important reason for your institution to initiate undergraduate student 
assessment 

b. have increased your institution's involvement in undergraduate student 
assessment 

c. have not been a factor in your institution's undergraduate student assessment 
activities 

d. have had a negative inlluence on your institution's undergraduate student 
assessment activities 

Your institution's regional accreditation agency requires (check ah that apply): 



D/R 

50 

77% 

18% 

0.7% 



All 

64% 

79% 

12% 

0 . 9 % 







D/R 


All 


a. 


evidence that a student assessment plan or process is in place 


83% 


90% 


b. 


intended institutional uses of student assessment information 


68% 


73% 


c. 


results of student assessment 


60% 


66% 


d. 


evidence of actual institutional use of student assessment information 


70% 


77% 


e. 


unfamiliar with regional accreditation requirements for student assessment 


9% 


5% 



O . 
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C. External Sources of Support for Assessment 



ERIC 



Has your institution received external grants to improve undergraduate student assessment practices I'rom any of 
the following? (check aU that apply): 

D/R All 

U)% 69r 

59c 19i 

19c 19 

1 1 9r 69; 

7495^ 799; 



FIPSE 

other federal agencies (please identify): . 
a state incentive program 



private foundations or corporate sources (please identify): 
no known external firants received 



Has your institution used any of the following student assessment services offered by the following 
postsecondary organizations? (check aJI services that apply for each type of organization): 



Student Assessment Service Used 



Type of Postsecondary Organization 


Not used or not 


Consultation 


Asscsstuent 


Training 


Publicatiotts 
or re.'^earch 




available 


services 


conferences 


workshops 


re pons 


a. Professional associations (Institutional, 


D/R: 24% 


17% 


629c^ 


38 %r 


63%r=^ 


disciplinary, or administrative) 


All: 29% 


139E 


5 1 % 


32% 


51% 




D/R:37% 


14% 


31%^ 


28 %r 


40 %r 


b. Regional accrediting association 


All: 30%: 


19% 


41% 


32%- 


45% 




D/R:68%^ 


10% 


19% 


15%: 


16%: 


c. State-level agency 


All: 54% 


14% 


26%' 


22%' 


22%’ 




D/R:569fc 


14% 


299c 


11%: 


25 %r 


d. Consortium of institutions 


All: 53%r 


\y7o 


30%' 


18%: 


20%: 


IV. Academic Management Policies 


and Practices 


for Student 


Assessment 







Institutions have a wide array of formally organized policies, activities, and procedures intended to enhance or support 
the collection and use of undergraduate student assessment information. The following policies and practices have 
been identified in many institutions. 

For Questions a through D, indicate whether the following policies or practices exist at your 

INSTITUTION. 



A. Resource Allocation for Student Assessment (check ah that apply): 

1. An explicit operating budget allocation is made to support student a.ssessment. 

2. An academic budget process that considers student performance indicators in resource 

allocation to academic units. 

3. An academic budget process that compares academic units on student performance 

indicators and allocates resources competitively. 

4. An academic budget process that rewards academic units for improvement ba.sed on 

their own past student performance indicators. 



D/R 


All 


2^9 9c* 


49%) 


2\9c 


23%) 


\9c 


2%) 


59c 


3%) 
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B. Student Assessment Information System (chock uH ihai apply): 





D/R 


All 


1. Key student assessment activities have been scheduled into the academic calendar. 

2. A computerized student information system which includes student perfomiance 


43 


57% 


indicators. 


27% 


2S% 


3. Student information system tracks individual students from application through 






graduation. 


50% 


42% 


4. Student assessment database integrated with faculty, curricular. & financial databases. 


6 % 


10% 


C. Access to Individual Student Assessment Information (check aU that apply): 




Student assessment information on individual students is available to. 


D/R 


All 


1. Institutional research, assessment or evaluation professionals 


69% 


76% 


2. Senior academic administrators 


55 %* 


72% 


3. Department chairs or academic program administrators 


62%=^ 


73% 


4. Student affairs professionals 


44 %* 


58% 


5. Faculty advisors 


56%=^ 


66% 


D. Distribution of Student Assessment Reports and Studies (check aU that 


apply): 




Student assessment reports and studies or appropriate summaries are regularly distributed to; 






D/R 


All 


1 . Students 


18% 


19%- 


2. Faculty 


50 


67% 


3. Academic administrators 


85% 


86% 


4. Student affairs professionals 


57% 


58% 


5. Employers 


6% 


5%' 


6, The general public 


13% 


8%> 



For Questions E through H, use the following scale to indicate the extent to which each ok the 

FOLLOWING POLICIES AND PRACTICES EXIST AT YOUR INSTITUTION (Circle one number lor each ilem). 





Not 


D(Mie in 


Dcme in 


Done in 


Done 




clone 


a few 


SCMllC 


many ' 


i n 


E. Student Policies on Student Assessment 


at ail 
1 


depis. 

2 


depis. 

3 


depis. 

4 


most 

depis. 

5 






D/R 




All 




1. Students are required to participate in student assessment 
activities 

2. Incentives are provided to encourage students to 




3.15^ 




3.77 




participate in student assessment activities 




1.85 




1.87 




3. Information regarding the purpose and uses of student 
assessment is provided to students 

4. Students are provided with individual feedback regarding 




3.11* 




3.52 




their own student performance results 




2.62* 




3.21 
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F. Professional Development 



D/R 



All 



1. Faculty are required to learn about or receive training on 
student assessment 


1.66* 


2A1 


2. Funds for faculty to attend or present at professional 
conferences on student assessment are available 


2.37* 


. 3.08 


3. Workshops, seminars, or consultative services for 
faculty on the use of student assessment in course 
design or instruction are offered 


2.65 


2.90 


4. Assistance for faculty in the form of paid leaves, 
stipends, mini grants or course reduction to improve use 
of student assessment is provided 


1.88 


2.00 


5. Workshops and seminars for department chairs, deans, 
and other academic administrators to improve use of 
student assessment in their unit is provided 


-X- 


2.35 


6. Student affairs staff are required to learn about or 
receive training related to student assessment 


1.98 




7. Student assessment workshops for student affairs 
administrators are provided 


2.01 


1 TO 



G. Faculty Evaluation and Rewards 



I. Faculty evaluation for promotion considers evidence of 
student performance in their classes (not just student 


D/R 


All 


teaching evaluation) 

2. Faculty evaluation for annual salary and merit increases 


1.74 


1.84 


incorporates evidence of student performance 
3. Faculty scholarship on or innovative uses of student 
assessment is considered in promotion, tenure, or salary 


1.69 


1.36 


reviews 

4. Faculty willingne.ss to use or to participate in student 
assessment activities is considered in faculty promotion. 


2.10 


2.01 


tenure, or salary reviews 

5. Faculty receive public recognition or awards for 


1.79 


1.99 


innovative or effective use of student assessment 
6. Faculty hiring process considers experience or skill in 


1.48 


1.58 


student assessment 

7. Faculty are encouraged to asse.ss student learning in their 


1.42 


1.68 


classes 


3.35* 


3.99 



• er|c 
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98 



H. Academic Planning and Review 

Your instiiuiion incorporates student performance data into 
the following processes: 

I. Academic department or undergraduate program planning 
or review 

2. General education or core curriculum review 

3. Course-level review and development 

4. Review and planning for student academic support 
services 



Not 


Done in 


Done in 


Done in 


Done 


done 


a fev\- 


some 


many 


in 


at all 


depts. 


depts. 


depts. 


most 


1 


-) 


3 


4 


depts. 

s 



D/R 


All 


3.33^ 


3.67 


3.10^ 


3.55 


2.9 2^ 


3.36 


2.77* 


3.09 



V_j Impacts of Student Assessment 

A. Decision Making 



To what extent has the use of information available from your undergraduate student assessment process innuenced 
the following actions? (circle one number lor each item): 



o 

ERIC 





No action or 


Action taken. 


Action taken. 


Action taken. 




inti ucncc 


data not 


data .>ioincwiKit 


(tat a vejy 


Institutional Actions 


unknown 

i 


intluontiai 


inllucntial 


inlluemial 

4 






D/R 


All 




1. Revising your undergraduate academic 

mission or goals 

2. Designing or reorganizing academic 




1.70* 


2.06 




programs or majors 




2.35 


2.54 




3. Designing or reorganizing student affairs 










units 




1.96 


1.91 




4. Allocating resources to academic units 

5. Modifying student assessment plans. 




1.62 


1.81 




policies, or processes 




2.41 


2.61 




6. Deciding faculty promotion and tenure 




1.34 


1.46 




7. Deciding faculty salary increa.ses or rewards 

(release time, travel funds, etc.) 

8. Revising or modifying general education 




1.33 


1.39 




curriculum 




2.30 


2.47 




9. Creating or modifying student out- of-cla.ss 










learning experiences (e.g. internships, 
service learning) 




2.10 


2.14 




IG. Creating or modifying distance learning 










initiatives 




1.58 


1.72 




1 1. Modifying instructional or teaching 










methods 




2.34 


2.47 




12. Modifying student academic support 










services (e.g. advising, tutoring) 




2.62 


2.56 
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B. Institutional Impacts 



Have you monitored the following institutional indicators and been able to document the impact of student 
assessment information on them? (circle one number for each item): 



Not monitored, 
do not 

Internal Impacts know 



1 . Affected campus discussions of 

undergraduate education 

2. Contributed to faculty 

satisfaction 

3. Contributed to faculty interest in 

teaching 

4. Led to changes in instructional 

or teaching methods used 

5. Contributed to student 

satisfaction 

6. Affected student retention or 

graduation rates 

7. Affected student grade 

perlbrmance 

8. Affected student achievement on 

external examinations (e.g. 
professional licensure, GRE) 

External Impacts 

9. Affected student applications or 

student acceptance rates 

10. Affected allocation or share of 

state funding 

1 1 . Affected evaluation from 

regional accreditation agency 

12. Affected private fund-raising 

results 

13. Affected success on grant 
applications 

14. Affected communication with 
external constituents 

15. Affected institutional reputation 
or image 

Note: * denotes difference greater than or equal to .? 



Monitored. 

negative 

impact 

2 


Monitored, 
no known 
impact 


Ntoniti'red 

positive 

impact 

4 


D/R 


All 




2.13 


2.28 




1.41 


1.69 




1.67 


1.88 




2.29 


2.45 




1.93 


2.03 




2.04 


2.20 




1.80 


1 .95 




1.85 


1.97 




D/R 


All 




1.48 


1.48 




1.62 


1.46 




2.50 


2.55 




1.42 


1.42 




1.48 


1.65 




1.76 


1.75 




1.72 


1.94 





or 10% from average of all institutions. 



o 
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VI. Further Studies - Optional 



This page will be removed from ihe questionnaire before it is processed and completion of it is optional. However, 
we would like to know more about your institution's experience with student assessment and we would like to be 
able to respond to you personally with a follow up report. 



Within the next year several institutions will be invited to participate in a more intensive study of the impacts of 
their student assessment practices and policies. Would you be interested in participating in a case study ? 



yes 

possibly 

no 



If you are interested, we would appreciate any additional information regarding your student assessment practices that 
you believe would be of interest to other institutions. If you believe your approach to student assessment c^r its 
impacts are unusual, please describe it briefly (or enclose 'a report you think captures your experience). 



Please provide your name and address if you are interested in receiving a personal summary report of this survey. 

Name; 

Title; ^ 

Institution: 

Address: 



Phone: 

E-Mail: 



Thank you for taking the time to complete this instrument. 
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Appendix B 



Construction and 
Content of V ariables 



o 
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Variable 

Institutional Characteristics 



Definition 



Enrollmem 


Single item. Reflects number of students enrolled in institution. Data from 1995 
Integrated Postsecondar>' Education Data System (IPEDS), 


Control 


Single item. (1 = public: 0 = private). Data from IPEDS. 


Institutional type 


Four dummy-coded single items. Rellects the instituticMi's Carnegie l\ pe. 
(Associate of Arts. Baccalaureate. Doctoral, and Research. Master's institutions 
was the omitted category .) Data from IPEDS. 



External Influences on Student Assessment 



State initiative 


Single item. Rellects whether the state's assessment initiatives were guided by 
legislative or other means (1 = no state plan: 2 = state policy: 3 = state statute: 4 
= combination of policy & statute). Data from SAS. 


State approach 


Single item. Reflects whether states mandate common indicators and (Uileomes ( 1 
= no indicators or outcomes: 2 = institutional specific: 3 = common for .some: 4 
= common for all). Data from SAS. 


Accrediting association 


Five dummy-coded single items. Rellects the institution's regional accivditaiion 
association membership (Middle States: North Central: New England: Southern: 
Western. Northwest region was the omitted region). Data from IPEDS. 


Development of state plan 


Single item. (Ill A 1) Rellects how state plan for student assessment was 
primarily developed (1 = state: 2 = joint consultation between slate and 
institution: 3 = no stale plan or requirement). 


State influence 


Four single items. (Ill A 2 a-d) Relied the inlluence of stale requirements on the 
institution.s as.sessment activities: a = important reason to initiate student 
assessment: b= increased institution's involvement in as.sessment: c = have not 
been a factor in assessment activities: d = have been negative inlluenee on 
assessment activities (I = yes: 0 = no). 


State reporting requirements 


Four single items. (Ill A 3 a-d) Relied the stale's reporting requirements; a = 
evidence that assessment plan is in place: b = nieasurement of stale mandated 
indicators: c = use of institutionally devised indicators: d = evidence of 
institutional u.se of assessment information (1 = yes: 0= no). 


State review methods 


Four single items. (Ill A 4 a-d) Relied the methcHl used by state to review the 
institutions assessment activities: a = reviewed by stale officials: b = reviewed 
using external reviewers: c = required institutional self-review: tl = no review 
occurred ( 1 = yes: 0 = no). 


State review criteria 


Five single items. (Ill A 5 a-e) Relied the processes included in the slate review 
of the institutions assessment activities: a = review of institutions process itself; 
b = compare student performance record with past record; c = compare student 
performance record with peer institutions; d = compare student performance record 
with other in slate; e = other ( 1 = yes; 0 = no). 



Variable 

Accrediting influence 



Accrediting reporting 
requirements 



Definition 

Four single items. (Ill B 2 a-d) Rcncci llic inllucncc ot regional accreditation 
agency requirements on the institutions assessment activities: a = important 
reason to initiate student assessment: h = increased institution's involvement in 
assessment: c = have not been a factor in assessment activities: d = have been 
negative influence on assessment activities (1 = yes: 0 = no). 

Five single items. (Ill B 3 a-e) Relleci the regional accreditation agency reporting 
requirements: a = evidence that assessment plan is in place: b = intended uses of 
assessment information; c = results of as.se. ssment: d = evidence ol actual 
institutional use of assessment information; e = unfamiliar with regional 
accreditation requirements ( 1 = yes; 0 = no). 



External sources of support Five single items. (Ill C I a-e) Reflect the .sources of support received to improve 

student assessment practices: a = FIPSE: b = other federal agencies: c = state 
incentive program: d = private foundation or corpe^rate source: e = no known 
external grants (1 =yes;0 = no). 

Use of external .services Four single items. (Ill C 2 a-d) Rellect the use ol services ottered by each o( the 

following type of posisecondary organization: a = professional a.ssociations: b = 
regional accrediting association: c = state- level agency: d = consortium of 
institutions. Respondents could choo.se from the t'ol lowing services offered by 
each organization: organization not used or not available: consul tat i(Ui .services: 
as.sessment conferences: training workshops: publications or research reports (1 = 
used: 0 = not used). 



Institutional Approach to Student Assessment 

Academic intentions Single item. (I A 1 ) Reflects extent to which institutions collect data on current 

student's academic intentions or expectations ( 1 = not collected; 2 = collected for 
some students: 3 = collected for many students: 4 = collected tor all students). 



Basic collciie-readine.ss skills Single item. (I A 2) Reflects extent to which institutions collect data on current 

student's college-readiness skills ( I = not collected: 2 = collected for some 
students: 3 = collected for many students: 4 = collected for all students). 



Cognitive as.sessment 



Affective a.s.sessment 



Four item factorially-derived .scale. (I A 3-6) Reflects the extent to which 
institutions collect data on current students' cognitive performance: competence 
in major field; general education competencies; higher-order cognitive skills: 
vocational or professional skills (1 = not collected: 2 = collected for .some 
students; 3 = collected for many students: 4 = collected for all students). Cronbach 
alpha = .7 1 . 

Three item factorially-derived .scale. (I A 7-9) Reflects the extent to which 
institutions collect data on current students' affective development and 
satisfaction: experiences and involvement with institution; satisfaction with 
institution; personal growth and affective development (1 = not collected; 2 = 
collected for some students; 3 = collected for many .students; 4 = collected for all 
students). Cronbach alpha = .68. 



Academic progre.ss Single item. (I A 10) Reflects extent to which institutions collect data on current 

student’s academic progress (1 = not collected; 2 = collected for .some students; 3 
= collected for many students; 4 = collected for all students). 









Variable 

Post-college assessmem 



Civic/social roles 



Timing of daia collcciion 



Number of insiruments 



Sludenl-ceniercd mclhods 



External methods 



Transcript analysis 



External examination 



Interviews of withdrawing 
students 



Definition 

Three item factorially-derived scale. (I A 1 L12.14) Rellects the extent to which 
institutions collect data trom former students: vocaticmal or professional 
outcomes: further education: satisfaction and experiences with institution after 
leaving ( 1 = not collected: 2 = collected for some students: 3 = collected for many 
students; 4 = collected for all students). Cron bach alpha = .S3. 

Single item. (I A 13 ) Rellects extent to which institutions collect data on former 
student's civic or social roles in the community ( 1 = not collected: 2 = collected 
for some students: 3 = collected for many students: 4 = cc41ected for all students). 

Nine item additive index. (I A 1-9) Rellects when institutions collect data ( 1 = 
not collected: 2 = collected at one point in time: 3 = collected at entry and w hile 
enrolled, or while enrolled and at exit: 4 = collected at entry and at exit: 3 = 
collected at entry, while enrolled, and at exit). • 

Nine item additive index. (1 B 1-9) Rellects student assessment instruments 
(institutionally developed, state provided, and commercially available) used by 
institution to collect ten types of assessment inftirmation: student plans c>r 
expectations: basic college-readiness skills: higher-tirder cognitive skills: general 
education competencies: competence in major: vocational or professional skills: 
personal grow'th and affective development: experiences or involvement with 
institution; satisfaction with institution (1 = instrument used: 0 = instrument not 
used). 

Four item factorially-derived scale. (1 C 1-4) Rellects the extent to which 
institutions use innovative or nontraditional assessment methods: performance in 
capstone courses: portfolios or comprehensive projects: observations of student 
performance: individual interviews or focus grtmps (1 = ntn used: 2 = used in 
some units: 3 = used in most units: 4 = used in all units). Cronbach alpha = .61. 

Two item factorially-derived .scale. (I C S-9) Rellects the extent t(> which 
institutions use as.sessment methtKls that data frtmi external constituencies: 
employer interview's or focus groups: alumni interview's or focus groups ( 1 = not 
used; 2 = used in some units: 3 = used in most units: 4 = used in all units). 
Cronbach alpha = .63. 

Single item. (I C 5) Reflects extent to w'hich institutions use transcript analysis 
to collect student as.sessment information (1 = not used: 2 = used in .some units: 

3 = used in most units: 4 = used in all units). 

Single item. (I C 6) Reflects extent to w'hich institutions use external 
exaniinations to collect student as.sessment informatitm ( 1 = not used: 2 = u.sed in 
some units: 3 = used in most units: 4 = u.sed in all units). 

Single item, (1 C 7) Rellects extent to which institutions use interviews with 
withdrawing students to colled student assessment information ( 1 = not used; 2 = 
used in some units; 3 = u.sed in most units: 4 = used in all units). 
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Variable 


Definition 


Student sub-populations 


Four single items. (I D 1-4) Rellect the u.se ol'differcnl assessment methods hu' 
the following different student populations: a = adult students; b = part-time 
students; c = minority students; d = distance education students ( 1 = different 
method; 2 = same method). 


Number of studies 


Nine item additive index. (I E 1-9) Reflects the number of studies institutions 
conduct on the relationship between aspects of students' institutional experiences 
and performance; course-taking patterns; exposure to different teaching metlK)ds; 
patterns of student-faculty interaction; extra-curricular activities; residehee 
arrangements; financial aid and/or employment; admi.ssion standards or policies; 
academic advising patterns; classroom, library and/or computing re.sources ( 1 = 
conduct study; 0 = do not conduct study). 


Number of reports 


Five item additive index. (I F 1-5) Reflects the levels of aggregation at whieh 
student assessment data are provided as reports: institution- wide; .schools or 
colleges: academic programs or departments; special populations or subgroups of 
students; by course or groups of courses ( 1 = report provided; 0 = report not 
provided). 



Organizational and Administrative Support Patterns 



Mi.ssion emphasis 


Three item additive index. (II A 1 a-c) Refleets institutions' mission statement 
emphasis on undergraduate education and its assessment: emphasi/.es execllenee in 
undergraduate education; idenlines educational outcomes intended for students; 
refers to student assessment as important activity (1 = yes; 0 = no). 


Internal purposes 


Four item factorially-derived score. (II B 3-6) Reflects the importance of internal 
institutional purposes for undertaking student assessment; guiding undergraduate 
academic program improvement; improving achievement of undergraduate 
students; improving faculty instructional performance; guiding resource allocation 
decisions (1 = no importance; 2 = minor importance; 3 = moderate importance; 4 
= very important). Cronbach alpha = .79. 


Accreditation purpo.ses 


Single item. (II B 1) Reflects importance of preparing for institutional 
accreditation .self-study as a purpo.se for undertaking student assessment (1 = no 
importance; 2 = minor importance; 3 = moderate importance; 4 = very 
important). 


State purpo.ses 


Single item. (II B 2) Reflects importance of meeting state reporting requirements 
as a purpose for undertaking student assessment (1 = no imponance; 2 = minor 
importance; 3 = moderate importance; 4 = very important). 


Administrative and governance 
activities 


Seven item additive index. (II C 1-7) Reflects the number of administrative or 
governance activities used by institutions to promote .student assessment; annual 
i n St i tut ion -wide as.sessmenl forums or seminars; rewards or incentives for 
administrators promoting u.se of assessment in unit; incentives for academic units 
to use assessment information; assessment workshops for administrators; board 
of trustees committee addresses asse,ssmeni: faculty governance committee 
addresses as.sessmenl; student representation on assessment committees (1 = yes; 

0 = no). 
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Variable 


Definition 


Administrative and faculty 
support 


Four item additive index. (II D 2-5) Rellects the degree to which chief executive 
officer, academic and student affairs administrators, and faculty support student 
assessment ( 1 = very unsupportive: 2 = somewhat unsupporti\e; = neutral or 
unknown; 4 = somewhat supportive; 5 = very supportive). 


Type of plan or policy 


Seven single items. (II E 1 a-g) Rellects the institutions plan or policy for 
student assessment: a = fomially adopted plan or policy requiring asse.ssment 
activities for ah academic units: b = formally adopted plan or policy requiring 
assessment activities for some acadetiiic units: c = tbriiiallv adopted plan or 
policy requiring ^academic units to develop their own assessment plan: d = 
formally adopted plan or policy stipulating institution-wide activities to be 
conducted by central committee, office, or officer, e = has no, formal plan or 
policy but academic units are encouraged to conduct their own assess iiicnt 
activities: f = is currently developing plan or policy: g = does not have an 
assessment plan or policy ( 1 = yes; 0 = no). 


Formal centralized policy 


Single item. (II E 1 a) Reflects institution has formal institutional plan or policy 
requiring specified student assessment activities of all academic units or programs 
(1 = yes; 0 = no). 


Institution-wide planning 
group 


Single item. (II E 2) Rellects institution has institution-wide group for student 
assessment planning and policy .setting (1 = yes: 0 = no). 


Breadth of assessment planning 
group 


Nine item additive index. (II E 3 a-i) Rellects the number of internal members 
included in the institution's as.sessment planning group: chief executive officer: 
academic affairs administrator(s)/stalT; student affairs administrators )/staff: 
institutional research administrator(s)/staff: academic review and evaluation 
administrator(s) /staff; student assessment administrator(s)/staff: faculty: students: 
other. 


Responsibility for planning 
group 


Seven single items. (II E 4 a-g) Rellect tlie internal members who have executive 
responsibility for the institution -wide grcnip responsible for planning or policy- 
setting for assessment: a = academic affairs administrator: b = student affairs 
administrator: c = institutional research officer: d = academic review and 
evaluation officer: e = student as.se.ssment officer: f = faculty member: g = other 
( 1 = yes: 0 = no). 


Approval authority 


Eleven single items. (II E 5 a-k) Reflect the internal members who approve any 
changes to institutions assessment plan or policy: a = board of trustees: b = chief 
executive officer: c = chief academic affairs tdllcer: d = chief student affairs officer, 
e = institutional re.search officer; 1 = academic review and evaluation officer: g = . 
student assessment officer: h = student government: i = academic .senate or other 
faculty committees: j = faculty union: k = other (1 = yes: 0 = no). 


Operating responsibility 


Eight single items. (II E 6 a-h) Rellect the internal members who have 
operational responsibility for the institution’s day-to-day assessment activities: a 
= academic affairs administrator; b = student affairs administrator; c = institutional 
research officer: d = academic review and evaluation officer: e = student 
assessment officer; f = faculty member; g = other: h = no one (1 = yes: 0 = no). 
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Variable 


Definition 


Reporting relationship 


Six single items. (II E 7 a-f) Reflect the individual to whom person with day-to- 
day responsibility reports: a = chief executive ofllccr; b = chief academic affairs 
officer: c = chief student affairs officer: d = institutional research officer: e = 
academic review and evaluation ofllccr: f = other ( 1 = yes: 0 = no). 


Conducted evaluation 


Single item. (II F 1 a-d) Reflects if institution has formally or informally 
evaluated its student assessment process ( 1 = yes: 0 = no). 


Evaluations elements 


Eight single items. (II F 2 a-h) Reflect the elements that were reviewed during the 
institutions as.sessment evaluation: a = student assessment plan or policies: b = 
structure and responsibility for assessment: c = achievement of intended 
objectives: d = reliability and validity of instruments and methcxls: e = quality of 
data analysis: f = use of information in institutional decisicm-making: g = 
problems encountered: h = comparison of costs and bene 11 ts ( 1 = yes: 0 = no). 



Assessment Management Policies and Practices 



Budget decisions 


Two item additive index. (IV A 3-4) Reflects lormal use of assessment 
information in the budget process: to competitively allocate resources among 
academic units; to reward academic units for improvement ( 1 = yes: 0= no). 


Computer support 


Three item additive index. (IV B 2-4) Re fleets institutional capacity to collect and 
manage student assessment information: computeri/ed student information 
system includes student performance indicators: student information system tracks 
individual students: student as.sessment databa.se integrated with other institutional 
databases ( 1 = yes: 0 = no). 


Acce.ss to information 


Five item additive index. (IV C \ -5) Reflects internal accessibility of a.s.se.ssmcnt 
information on individual students by: institutional research or assessment 
professionals: .senior academic administrators: department chairs or academic 
program administrators: student affairs professionals: faculty advisors ( 1 = yes; 0 
= no). 


Distribution of reports 


Six item additive index. (IV D 1-6) Reflects the number of constituent groups to 
whom student assessment reports are regularly distributed: students: faculty: 
academic administrators: student affairs professionals: employers: general public 
( 1 = yes: 0 = no). 


vStudent involvement 


Three item factorially-derived .scale. (IV E 1.3,4) Reflects the extent to which 
institutions have policies or practices to promote student involvement in 
assessment activities: inform students about assessment purpevses and u.ses: 
require students to participate in assessment activities: provide students with 
individual feedback on assessment results ( 1 = ntu done at all: 2 = done in a few' 
departments: 3 = done in some departments; 4 = done in many departments: 5 = 
done in most departments). Cronbach alpha = .69. 
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Variable 

Professional developmeni 



Siudeni affairs training 



Faculty evaluation 



Academic planning and review 



Institutional Uses of Student 
Educational decisions 



Definition 

Four item factorially-derived scale. (IV F 2-5) Rellecis existence ol prole.ssional 
development policies or practices on student asses.sment for faculty and academic 
administrators: provide funds for faculty to attend or present at asse.ssmeni 
conferences: offer student asse.ssment work.shops or consultation for faculty: 
provide assistance (e,g.. paid leaves, stipends, course reduel imi ) io improve 
faculty use of student as.sessmenl; provide student a.s.se.ssmeni workshi^ps for 
academic administrators (I = not done at all: 2 = done in a lew depart men is: = 

done in some departments: 4 = done in many deparimeiiis: 5 = done in most 
departments). Cronhach alpha = .77. 

Two item factorially-derived scale. (IV F 6-7) Rellecis existence ol prole.ssional 
developmeni policies or practices on student a.ssessmeni for student al lairs 
personnel: require assessment training for student affairs staff: provide student 
assessment workshops for student affairs administrators ( 1 = not done al all: 2 = 
done in a few departments: 3 = done in .some departments: 4 = done in many 
departments: 5 = done in most departments). Cronhach alpha = .S4. 

Five item factorially-derived scale. (IV G 1-5) Rellecis existence ol I acuity 
evaluation and reward policies and practices related to student a.ssessmeni: 
promotion evaluation considers evidence of student performance: salary evaluation 
considers evidence of student performance: promotion, tenure or salary reviews 
consider faculty participation in student a.ssessmeni: promotion, tenure or salary 
reviews consider scholarship on asse.ssmeni: public recognition or awards lor 
faculty use of student assessment ( 1 = not done al all: 2 = done in a few 
departments: 3 = done in some departments: 4 = done in many departments: 5 = 
done in most departments). Cronhach alpha = .77. 

Four item factorially-derived scale. (IV H 1-4) Rellecis the incorporation ol 
student asse.ssmeni data into academic planning and review proce.s.ses for: 
academic deptirlmenis or undergraduate programs: general education or core 
curriculum: cour.ses: student academic support .services ( 1 = not done al all: 2 = 
done in a few departments: 3 = done in some departments: 4 = done in many 
departments: 5 = done in most departments). Cronhach alpha = .S4. 

Assessment 

Ten item factorially-derived .scale. (V A 1-5. 8-12) Rellecis the inlluence ol 
student asse.ssmeni information in educational decisions: revi.sion of undergraduate 
academic mi.ssion or goals: designing or reorganizing academic programs or 
majors: designing or reorganizing student affairs units: allocating re.sources to 
academic units: modifying student assessment plans, policies, or processes: 
revising or modifying general education curriculum: creating or modifying student 
oul-of-class learning experiences: creating or modifying distance learning 
initiatives; modifying instructional or leaching methods: modifying student 
academic support services ( I = no action or inlluence known: 2 = action taken, 
data not inllueniial; 3 = action taken, data .somewhat inllueniial; 4 = action taken, 
data very inllueniial), Cronhach alpha = .83. 
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Variable 

Faculty decisions 



Definition 

Two item raclorially-dcrivcd scale. (V A 6-7) Reneeis the inlluence of student 
assessment information in faculty decisions: deciding faculty promotion and 
tenure: deciding faculty salary increases or rewards (1 = no action or inlluence 
known: 2 = action taken, data not inlluential: 3 = action taken, data .somewhat 
inlluential; 4 = action taken, data very inlluential). Cron bach alpha = .7S). 



Institutional Impacts of Student Assessment 



Faculty impacts 



Four item factorially-derived scale. (V B 1-4) Rellects student a.s.sessment impacts 
on faculty: affected campus diseu.ssions of undergraduate education: contributed to 
faculty satisfaction: contributed to faculty interest in teaching: led to changes in 
teaching methods used (1 = not monitored, do not know: 2 = monitored, negative 
impact: .3 = monitored, no known impact: 4 = monitored, positive impact). 
Cronbach alpha = .79. 



Student impacts 



Four item factorially-derived scale. (V B 5-8) Rellects student assessment impacts 
on students: contributed to student satisfaction: affected student retention or 
graduation rates: affected student grade perlormanee: alleeted student achievement 
on external examinations ( 1 = not monitored, do not know: 2 = monitored, 
negative impact: 3 = monitored, no known impact: 4 = monitored, positive 
impact). Cronbach alpha = .82. 



External impacts 



Seven item factorially-derived scale. (V B 9-15) Rellects student assessment 
impacts on external constituents: affected student applications or acceptance rates: 
affected allocation or share of state funding: affected evaluation from regional 
accrediting agency: affected private fund-raising results: affected success on grant 
applications: affected communications with external constituents: affected 
institutional reputation or image ( 1 = not monitored, do not know: 2 = 
monitored, negative impact: 3 = monitored, no known impact: 4 = monitored, 
positive impact). Cronbach alpha = .82. 



' Assessment of Teaching and Learning For Improvement and Public Aecountabilitv: State Gcn'ernimi. Coordinating 
Board and Regional Accreditation Associatioti Policies and Practices (Cole et al.. 1997) 
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